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Editor’s Introduction 



Robert J. Samuelson, a columnist 
for the Washington Post, talks about 
American unhappiness in the midst of 
American success in his new book The 
Good Life and Its Discontents:The 
American Dream in the Age of Entitle- 
ment. He claims we transformed the 
American Dream into the American 
Fantasy. A reasonable demand for 
progress, he says, has become an ex- 
cessive demand for perfection. Conse- 
quently, “our attitudes are shaped more 
by unattained ambitions than actual 
achievements.” He insists that by most 
standards American society is better off 
than a half century ago. Does this au- 
thor understand what’s really happen- 
ing to American society? There is no 
question that many facets of American 
life have grown better over the past fifty 
years. But the results of competent re- 
search lead us to have some qualms 
about our overall “progress” as an 
egalitarian social democracy. 

Writing in the New York Times op-ed 
column in January, 1996, author of The 
End of Affluence, Jeffrey Madrick, has 
a slightly different perspective on soci- 
etal history. He acknowledges that al- 
though the income gap has widened 
and real wages fallen, most people are 
better off than their grandparents or 
great-grandparents. But he maintains 
that this comparison overlooks the sig- 
nificant historical shift in the economy 
and public attitudes that provides a 
reasonable rationale for the anxiety re- 
flected in American public opinion 
polls about the state of the union. 

Most workers did not live as well be- 
fore World War II as they do now, he 
says. But over the course of their lives 
they made extraordinary gains in their 
standard of living. In this context, Mad- 
rich points out that “exceeding one’s 



expectations defined what it meant to 
be American. ... It empowered people 
to think they could shape their own 
destiny.” However, in the past twenty 
years the rate of economic progress 
has slowed beyond any era since the 
Civil War. The output of goods and ser- 
vices produced per hour of work has 
also slowed. Furthermore, although 
rapid growth tends to benefit everyone, 
slow growth apparently does not. Given 
that the rate of economic growth and 
productivity are the main determinants 
of our living standards, these changes 
are shaping a new public malaise. 
Madrich concludes that this is a signifi- 
cant reversal of our national experi- 
ence and buoyancy. 

This blend of optimism and pessi- 
mism in the mid-1990s is affecting the 
economic, socio-cultural, political and 
value environment of this generation of 
adolescents and young adults, influ- 
encing their life chances, framing their 
attitudes and behavior. 

In this issue of Evaluation Forum we 
focus on American youth, and on the 
historical and contemporary forces that 
are creating a different society for them 
than the one earlier generations have 
experienced. This is a changed society, 
in which young people will become so- 
cialized, will mature, will seek to 
achieve developmental tasks, and will 
become the decisionmakers of the so- 
ciety’s future. 

Since we can address only a few of 
the many issues relevant to the matu- 
ration of youth into adult roles, we fo- 
cus in the Features Section on the 
darker side of American society -that 
is, on increasing economic insecurity 
and inequality - since many of our 
youth education, training and employ- 
ment programs respond to such basic 
societal problems. 



American Social 
Policy 

Important pieces of the fabric of 
complex designs against which youth 
issues must be analyzed are provided 
in Ronald B. Dear’s article on Social 
Welfare Policy: 7 Yends and Issues in the 
new three-volume Encyclopedia of So- 
cial Work published in 1995. Dear is a 
specialist in social welfare policy on the 
faculty of the University of Washing- 
ton’s Graduate School of Social Work, 
but his exposure to policy issues is not 
limited to academia. As the liaison be- 
tween the University and the Washing- 
ton State Legislature, and later as the 
chair of the University’s Faculty Senate, 
Dear has experienced up close the 
machinations involved in developing 
and implementing policy in a highly po- 
liticized environment. Never wavering 
from a dual interest in root causes as 
well as social policy strategies, his anal- 
yses are driven by the need to under- 
stand the underlying forces shaping so- 
cial welfare policy in the United States. 
This commitment helps frame a broad 
approach to youth issues, particularly 
given the increase in youth poverty and 
its correlates. 

Beginning with a definition and in- 
terpretation of social welfare policy, 
Dear gives us an historical perspective 
on American social policies that have 
attempted to support equity and justice 
for all citizens. He identifies the major 
influences and trends that have helped 
form such policies, and suggests proba- 
ble future directions. His analysis is 
concerned with the entire public social 
welfare system, including income sup- 
port, health care, education, employ- 
ment and training, and social services. 
He contends that youth problems de- 
manding policy action must be concep- 
tualized comprehensively, within the 
broader context of the larger social wel- 
fare system and the society. 

What Is Social Policy? 

The author explains social policy as a 
set of principles, plans, procedures and 
courses of action that set parameters 
around government action — what gov- 
ernment can and cannot do. These are 
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expressed in legislation, administrative 
rules and regulations, judicial decrees 
and organizational agendas. Social poli- 
cy involves basic assumptions and 
guidelines that determine the form gov- 
ernment activity takes in various sub- 
stantive areas. In this sense, Dear 
views public social policy as a tool or 
mechanism used by government for allo- 
cating limited resources, based on sev- 
eral premises: 

■ Public economic resources, as well 
as most natural resources, are almost 
always in limited supply. 

■ Individuals and societies have almost 
unlimited needs. 

■ Resource allocation processes are 
never fully equitable or just. Societ- 
ies have built-in unfairness. 

■ No consensus exists about how eq- 
uity and justice in resource distribu- 
tion are to be achieved. 

The author concludes that disagree- 
ment and conflict over the values to be 
honored in allocation processes are in- 
evitable. Since compromises involved in 
making allocation decisions tend to 
foreclose consideration of other options, 
Dear feels it is essential that the proba- 
ble impact of particular value tradeoffs 
be evaluated. In saying so, he is inter- 
ested in what forces are most likely to 
contribute to progressive social change. 

Among the influences affecting so- 
cial democratic change, Dear highlights 
economic scarcity in the public sector 
distrust of government, lack of agree- 
ment on a politically-sustainable social 
policy agenda , the absence of a political 
vision and the will to support innova- 
tive approaches, and the territoriality 
and pressures exerted by special interest 
groups. While these forces have compli- 
cated the development of remedies to 
significant social problems in the U.S., 
Dear reminds us that since World War 
11 we have been moving steadily in the 
direction of European “welfare states,” 
though with considerable ambivalence 
and hesitation. 

A Selected History of Attempts to 
Understand and Document Major 
SocialTrends 

Dear highlights Hoover’s appoint- 
ment of William F. Ogburn, a noted so- 
ciologist and Director of Research for 
the President’s Committee on Social 
Trends. Recent Social Trends, the mam- 
moth report of the committee, was per- 
haps a defining moment in American 
social welfare policy development. Og- 

Q 




burn’s work had supported the conclu- 
sion that social trends develop a force- 
ful momentum of their own and there- 
fore resist efforts to reverse them. The 
Committee’s report reiterated that posi- 
tion, emphasizing the trend toward 
greater government control in the inter- 
est of securing the common good. The 
report also encouraged government to 
collect, analyze and use quantitative 
time series data as a support for policy 
development and planning. This 
launched a historical effort to study the 
relationship between social trends and 
emerging social problems. 

Thirty years later there was another 
surge of interest in social trend analy- 
sis. Dear refers to a 1964 report by the 
U.S. Department of Health, Education 
and Welfare (HEW) that demonstrated 
how demographic trends and trends in 
education, employment, income and 
health care could converge to produce 
new social problems. A subsequent 
HEW report in 1969 attempted to show 
the utility of an ongoing analysis of so- 
cial indicators for identifying the evolu- 
tion of major social trends. Daniel Bell, 
co-chair of the government’s Panel on 
Social Indicators, viewed the report as 
the first bona fide effort to measure the 
performance of the society in meeting 
human needs. 

Unfortunately the Vietnam War and 
reduced social spending diverted atten- 
tion from this approach to social policy 
development. Interest in an annual so- 
cial welfare “report card” waned. And 
as the movement toward a more re- 
sponsive and generous government so- 
cial welfare policy slowed, this in itself 
became a major new social trend. Dear 
makes the case that the twenty years 
between 1981 and 2001 may someday 
be perceived as a turning point in social 
welfare policy, an era of increasing but 
less well met needs, of lost social wel- 
fare opportunities, and of dampened 
social innovation. 

The belief that government is the 
main institution responsible for the 
well-being of less fortunate citizens, 
represented by the New Deal and Great 
Society programs, was rejected in the 
Reagan-Bush period. Dear points out 
that conservative writers such as Mur- 
ray (Losing Ground) and Gilder (Wealth 
and Poverty) expounded a “new” theo- 
ry, that government social programs ac- 
tually increased rather than ameliorated 
social problems. Supply-side economic 
theory, Dear says, only confirmed the 
notion that shrinking the size of govern- 
ment and precluding its intrusion into 



the resource allocation process would 
solve unemployment and poverty. 
Reagan’s New Federalism converted 
such theories into pragmatic action by 
consolidating programs, reducing the' 
funding, decentralizing, and, whenever 
possible privatizing. “Devolution” was 
expected to encourage more efficiency 
and effectiveness and to interfere less 
with economic productivity and devel- 
opment. Block grants became one of 
the strategic mechanisms for acting on 
these premises. 

Using federal budget data, the Cen- 
ter on Budget and Policy Priorities 
showed that between fiscal years 1981 
and 1987 Community Development 
Block Grants were reduced by 39%, 
Community Services Block Grants by 
36%, and Social Services Block Grants 
by 28% (after inflation). The nation’s 
principal source of federal funding for 
child care and child protective services 
was Title XX of the Social Security Act, 
passed in 1975. Title XX required state 
matching funds, targeted funds to low 
income citizens, and mandated public 
participation in the development of 
state plans and social services. When 
folded into a block grant in 1981, these 
requirements were removed, social ser- 
vice funds were cut, and most report- 
ing ended. Also there were sharp re- 
ductions in transfer payments and 
in-kind benefits, and more restrictive el- 
igibility requirements. For example, in 
just two years Unemployment Insur- 
ance was reduced 17.4%, child nutri- 
tion programs by 28%, Food Stamp ex- 
penditures by 13.8%, and AFDC by 
14.3%. 

The author sees the combination of 
Reagan’s tax cuts and increased de- . 
fense expenditures during a recession- 
ary period as the beginning of our cur- 
rent obsession with federal revenue 
shortfalls. The annual federal deficit 
and debt soared, growing even larger 
under Bush. In the context of deficits 
and debts of unprecedented magni- 
tude, significant new social welfare ini- 
tiatives lost support. Moreover, nearly 
all existing federal social programs be- 
came suspect and vulnerable to reduc- 
tion, consolidation or elimination. Nev- 
ertheless, Dear reminds us that the 
large social insurance programs, such 
as Social Security and Medicare, sur- 
vived, leaving the core of the public 
American social welfare enterprise in- 
tact. However, the 1994 elections repre- 
sented the first major effort to revise 
this core, with the purpose of propos- 
ing a very different approach to re- 
source allocation. 
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Important Social and Economic 
Trends Affecting American Social 
Welfare Policy 

Dear emphasizes an array of changes 
in American society that have influ- 
enced the way we view social problems 
and try to solve them. 

Economic Change 

Dear agrees with most social policy 
experts that although the U.S. gener- 
ates approximately $6 trillion per year 
in wealth, and has the world’s highest 
level of productivity, a large segment of 
the population does not share in this af- 
fluence. Labor productivity has slowed 
dramatically and real earnings have de- 
clined. Both low-technology and high- 
technology products once manufac- 
tured in the U.S., supplying jobs for 
U.S. workers, are now frequently man- 
ufactured overseas and imported back 
into the U.S. Meanwhile, improved effi- 
ciency through automation has led to 
job loss. And many of the new jobs cre- 
ated pay lower wages than those lost in 
manufacturing. Dear says the 1970s 
and 1980s were characterized by low 
growth in labor productivity, company 
downsizing, plant closures, extensive 
loss of the better-paying manufacturing 
and factory jobs, sharp reductions in 
agricultural labor, costly health care, 
high inflation, and increasing taxes on 
the middle class. These factors, he pro- 
poses, seriously stagnated the incomes 
of middle and lower class Americans. 
Unfortunately, these trends continue. 

Technological Change 

Many jobs, Dear points out, now re- 
quire computer literacy. He quotes Pe- 
ter Drucker's prediction in Post-Capital- 
ist Society that “the basic economic 
resource... is no longer capital, nor nat- 
ural resources... nor labor. It 
is... knowledge.” The advent of the in- 
formation era has changed the nature of 
jobs. 

Education 

Dear believes the U.S. is moving to- 
ward two separate sub-societies, one 
consisting of well-educated and trained 
professional, technical and managerial 
workers; another made up of less well- 
educated and less skilled low-paid 
workers. He refers to former Secretary 
of Labor Robert Reich’s pronouncement 
that 20% of the U.S. population com- 
petes successfully socio-economically 
while 80% finds itself falling behind. 
Education is the key to economic ad- 



vancement. Although a larger propor- 
tion of Americans attend college than 
in most other countries, Dear cites evi- 
dence that lower-income and some mi- 
nority young people are considerably un- 
derrepresented in American higher 
education. This underrepresentation 
may be due to a number of factors, 
such as inadequate preparation, high 
cost, inaccessibility, discrimination, or 
the pessimistic view on the part of 
these youth that no strategy is likely to 
lead to mainstream economic success. 

Demographic Change 

Between 1930 and 1975 the popula- 
tion of the world doubled. Between 
1930 and 2000 the world's population 
will have tripled -- it is projected to 
reach 6 billion by 2000 and 8.5 billion 
by 2025. At the current growth rate, the 
population will increase as much in 
twelve years as it did in the first three 
million years since the beginning of hu- 
man existence. Within a thirty-month 
period, enough people are added to the 
world to populate another U.S., with 
ninety-five percent of this world popu- 
lation growth occurring in developing 
countries. 

Despite claims that natural resources 
remain plentiful and technology can 
solve environmental problems, Dear 
contends that resource distribution poli- 
cies will become a growing political 
challenge. He agrees with Paul 
Kennedy's thesis in Preparing for the 
TXventy-First Century that world popula- 
tion cannot be supported if current 
consumption attitudes, patterns and 
levels continue. 

In this country, although the popula- 
tion is growing at a slower rate than in 
developing countries, it is still faster 
than in the European Union, largely 
due to immigration. Minority popula- 
tions are growing more rapidly than the 
white population. Acknowledging that 
we share problems with developing 
countries in terms of income inequality, 
disparities in health care, high unem- 
ployment and dislocation, and social di- 
vision, he also refers to our uniqueness. 
Although the U.S. sustains less than 
five percent of the world’s population, 
it consumes a dramatically dispropor- 
tionate amount of its resources. 

Dear also gives attention to the age 
structure of the U.S. population. A pop- 
ulation increase of 125-150 million new 
residents in the next fifty years will, he 
suggests realistically, increase the need 
for new infrastructure development, 
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new home and business construction, 
and new organizational structures to 
support these changes. The aging of 
the “baby boom” generation and other 
demographic changes are important 
also. In 1940, only 6.8% of the popula- 
tion was over 65; by 1972 it was over 
12%. By 2030, over 20% of the U.S. 
population is projected to be over 65. 
The youth population, whose adolescent 
portion declined in the 1980s, is expect- 
ed to grow slowly through the 1990s. 

Those born between 1946 and 1964 
constituted up to that time the largest 
baby cohort in U.S. history, represent- 
ing 30% of the U.S. population in 1992. 
Dear views the size of this cohort as 
the predominant demographic event af- 
fecting social policy in this era. He 
places the generation born between 
1925 and 1942 between the politically 
powerful older generation and the nu- 
merically powerful baby boomers. Born 
in the Great Depression, the former, of- 
ten called “the silent generation,” is 
small comparatively. It escaped World 
War II, experienced “the forgotten 
war” in Korea, and entered the work- 
force during a period characterized by 
high levels of education, low unem- 
ployment and high socio-economic op- 
portunity. This generation, particularly 
middle-aged women, is currently caring 
for aging parents who are living into 
their 80s and 90s. Furthermore, the ra- 
tio of care givers to care receivers is in- 
creasing, Dear says, as there are pro- 
portionately fewer people age 50-64 to 
care for those 85 and older. 

The third layer of the “population 
sandwich” is the 80 million people age 
12 to 32. Numerically larger than the 
“baby boomers,” they tend to see 
themselves as maturing just as the soci- 
ety seems to be deteriorating, with up- 
ward mobility opportunities apparently 
decreasing. This disenchanted genera- 
tion is the most diverse in terms of 
backgrounds and orientations, Dear 
says, and the most tolerant of new def- 
initions of life styles, social norms and 
family forms emerging out of the civil 
rights, anti-war, and women’s move- 
ments and the sexual revolution. The 
social, economic and cultural differenc- 
es associated with the different eras ex- 
perienced by the three major segments 
of the U.S. population, have created un- 
derstandable strains in the develop- 
ment of social welfare policy and pro- 
grams that are responsive to and 
acceptable to all three groups. 

A parallel development analyzed by 
Dear is the change in the racial and eth- 
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nic composition of the population. In 

1992, 75% of the U.S. population was 
classified white, 13% black, 9% His- 
panic, and 3 % Asian Pacific Islanders. 
The proportion of ethnic minorities has 
gradually increased in the 1980s, as has 
the annual growth rate. The annual 
growth rate for whites was 1% but for 
minorities it was much higher — 1.5% 
for blacks, 2.2% for Native Americans, 
6.4% for Hispanics, and 6.62% for 
Asian Americans. The projection for 
2050, given rates of growth in the 
1990s, is that whites will constitute 
53% of the population, Hispanics more 
than 20%, blacks 16%, and Asian 
Americans 11%. This will create a more 
ethnically diverse population, but likely 
void of a commitment to the “melting 
pot” concept that distinguished our ear- 
lier blending of historical roots, religions 
and cultures into a single-language, sin- 
gle-culture society. 

Changes in American Family 
Norms, Structures and Life Styles 

Dear joins other experts in viewing 
the dramatic rise in the number of sin- 
gle-parent families as a major legacy of 
changing family patterns, linked inevi- 
tably with rising poverty. In 1970, 11% 
of all families were headed by a single 
parent, usually a woman. By 1990, 
nearly a quarter of all families were so 
structured. Dear looks for causes in the 
higher divorce rates, the lower propor- 
tion of remarriages, and the significant 
increase in never-married mothers. By 

1993, there was one divorce for every 
two marriages, on average. The number 
of young unmarried men and women 
had risen - by 1990, 63% of women 
age 20-24 had never married, compared 
with 28% in 1960; 79% of men in this 
age group had never married, compared 
with 53% in 1960. In 1960, one in 
twenty births was to an unmarried 
mother; in 1990 more than one in four 
births, and almost two-thirds of all 
black children were born to unmarried 
mothers. 

Dear also comments on the composi- 
tion of families, indicating the increase 
in reconstituted families and unmarried 
but co-habiting couples. He emphasiz- 
es changing family work patterns, with 
both spouses in 60% of two-parent 
families in the workforce in 1990, a sig- 
nificant change from 40% in 1970. 

Change in the Distribution of 
Income and Wealth 

D.ear shows that between 1977 and 
1988, the after-tax income of the poor- 
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est one-fifth of all households fell by 
10%. At the same time, the income of 
the top fifth increased by 34%. Accord- 
ing to the Center on Budget and Policy 
Priorities, the gain for the top level was 
a staggering 122%! Wages and salaries 
for this well-educated, highly skilled 
group more than doubled from 1977 to 
1988. Dear also alerts us to tax breaks 
for this group and the increase in capi- 
tal gain income. At the other end of the 
income continuum, AFDC benefits de- 
clined 45% from 1970 to 1993. The dis- 
parity between high and low income 
populations had become striking and 
disturbing. Furthermore, according to 
data available in 1966, trends toward 
income inequality are continuing into 
the 1990s. 

Beyond these figures. Dear tells us 
that the number of Americans in pover- 
ty climbed in 1992 for the third consec- 
utive year, higher than in any year 
since 1962. Approximately one in seven 
Americans were classified poor in 1992. 
Slightly over 30% of blacks, and close 
to 30% of Hispanics were determined 
to be poor, while the poverty rate was 
12% for whites and the same for Asian 
Americans. 

In 1 992, one-fourth of children under 
age six were poor. Close to 22% of youth 
under 18 were poor. Nearly half of all 
black children under 18 were poor. And 
there was an increase in the poverty rate 
for black married couples , which is sig- 
nificant in the context of poverty rates 
typically being much lower for couples 
than for single heads of households. 
One-half of all female-headed families 
with children were poor. The absolute 
level of poverty was also shocking. Al- 
most one-half of all American children 
in poverty lived in households with an 
income less than half of the official pov- 
erty level! Furthermore, these trends were 
in marked contrast to the lowering of 
the poverty rate for the elderly. 

Dear calls attention to the fact that 
information about these demographic 
trends has dispelled the notion that un- 
employment is always the primary cor- 
relate of poverty. Approximately 40% of 
the poor were employed in 1992. Ten 
percent had full-time, year-round jobs. 
A changing economy, moving rapidly 
from manufacturing to services and in- 
formation, has eroded high-paying low- 
er-skill employment, and increased 
competition for emerging high-income- 
generating jobs. And racism and sexism 
continue to limit the educational, train- 
ing and employment opportunities of 



women and minorities in many occupa- 
tional niches. 

Waning Faith in Government 

The U.S. was established by people 
who left oppressive authoritarian gov- 
ernments. Consequently the Constitu- 
tion incorporated a formal system of 
checks and balances to prevent the for- 
mation of a powerful and intrusive gov- 
ernment. Unusual emphasis was placed 
on the rights of states and individuals. 
But this diffusion of power, Dear sug- ‘ 
gests, has slowed the development of 
social welfare policies and programs. 
And other qualities of government have 
been barriers in responding to social 
need. Deficits, special interest group in- 
fluences, and shrinking revenues for 
new social expenditures have paralyzed 
government action, precluding new initi- 
atives, and an evaluation of the con- 
tinuing effectiveness of programs as a 
basis for positive change. 

The Impact of theTrends 

Dear concedes that some programs 
for the poor have received more fund- 
ing in the 1980s and 1990s, but he 
claims there is ample evidence that 
most anti-poverty efforts have not ex- 
panded consistent with economic 
growth, inflation and human needs. 

Not only income assistance but also 
employment and training and housing 
programs have been reduced, while tax 
benefits for upper income people have 
grown substantially. The author is par- 
ticularly interested in the effects of spe- 
cial income tax deductions, exclusions, 
preferential tax rates, credits and defer- 
rals of tax liabilities, that represent sub- 
stantial benefits to those with sufficient 
income to take advantage of such poli- 
cies.’ He reports that in 1994, tax bene- 
fits of these kinds cost the federal gov- 
ernment $256.4 billion. They are 
projected to cost $337 billion by 1998, 
considerably more than the projected 
federal deficit for that year. 

Perceived economic scarcity, tax 
welfare for the affluent, and the desire 
to shrink government and limit its con- 
trol has spawned a new “linguistic sys- 
tem,” — devolution, decentralization, 
deinstitutionalization, declassification, 
reconfiguration, consolidation , privati- 
zation, volunteerism. The broader “wel- 
fare state” view, in which benefits and 
services enhancing life quality and op- 
portunity are seen as rights of citizen- 
ship, was lost, Dear concludes. The 
general public had failed to under- 
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stand, or had preferred not to acknowl- 
edge, he believes, that almost all major 
social programs and policy victories had 
been accomplished at the federal level, 
often in opposition to less universalistic 
approaches at the state and local level. 
Neither did the public realize that the 
administration of these large programs, 
such as Social Security, were costing no 
more in 1992 than 1% of the benefit 
payments they distributed, a record 
even by corporate standards, Dear pro- 
poses. 

There are reasons for optimism, Dear 
insists, that national social welfare 
guarantees will not be totally disman- 
tled in the 1990s, but he sees a number 
of opposing forces at work which are 
influencing and will likely continue to 
direct social policy development away 
from the “welfare state” concept. And 
he is concerned about the constraints 
the trends he has analyzed place on 
progressive social planning: 

■ The decline in liberalism. 

■ The growing national debt and con- 
tinuing high deficits. 

■ Social program cutbacks due to inad- 
equate state and local revenue. 

■ Growing citizen distrust of govern- 
ment. 

■ A changing economic base. 

■ Rapid movement toward internation- 
al trading blocks. 

The Need for a Re-Invigorated 
Social Agenda 

Dear is not reticent when it comes to 



suggesting policy revisions. The conclu- 
sion of his chapter is bold, his proscrip- 
tion controversial in the present politi- 
cal environment (see Figure 1). 

Relationship of Dear’s Analysis 
to Youth Issues 

The economic, social, cultural and po- 
litical trends Dear identifies as key in- 
fluences on social policy in the U.S. 
have an impact on government social 
expenditures for youth education, train- 
ing and employment. Young people’s 
ability to acquire a level of knowledge 
and skills that affords choice and op- 
portunity in adulthood is critical to their 
future life quality, and that of their 
families and their own children. The 
trends of concern to Dear have created 
vastly different societal environments 
and socialization resources for different 
groups of youth in American society, 
substantially reducing life chances for 
an increasingly large proportion of the 
youth population. If greater equity is 
not achieved, Dear feels the disparities 
in opportunity will have serious reper- 
cussions for American society in the fu- 
ture in terms of disenchantment, isola- 
tion and alienation. His fears are well 
worth addressing. 



Editor’s Note 



For readers interested in more de- 
tail, please see “Social Welfare Policy” 
in the Encyclopedia of Social Work, 
19th Edition. Washington, DC: NASW 
Press, 1995. 



A Generational 
Perspective on 
Young Adults 

In the previous article, Ronald Dear 
provided an overview of recent demo- 
graphic changes affecting the formation 
of social policy. He emphasized the cul- 
tural divergence across four important 
generations represented in the current 
American population: the elderly, the 
middle-aged, the Baby Boomers, and 
the young adult children of the 
Boomers’ cohort. In this article we ex- 
plore further some of the differences 
between those maturing during and 
shortly following World War II and the 
young adults of the 1980s and 1990s. 

Qualifications to the Generational 
Comparison 

The commentaries summarized here 
focus on the student generations of the 
late 1940s-early 1950s, and the late 
1980s-early 1990s at a single liberal arts 
college, Oberlin College in northern 
Ohio. Long known for its high academ- 
ic standards and social consciousness, 
Oberlin students have been selected 
traditionally not only for their academic 
prowess but for their leadership quali- 
ties and their commitment to social and 
democratic values. The insights emerg- 
ing in a comparison of these two co- 
horts of Oberlin students make an im- 
portant contribution to understanding 
the changes that have taken place in 
society over the decades involved, and 
in the environments in which youth 
grow to maturity in the 1990s. Howev- 
er, the uniqueness of this single case 
study requires caution in generalizing 
the commentary to other American col- 
lege students and to non-college-bound 
American youth. 

The limited focus on Oberlin stu- 
dents is clearly outweighed by the 
knowledge and experience of the two 
individuals making the comparison - 
Nancy Dye, the President of Oberlin 
College and an historian, and Oberlin 
Professor Emeritus Albert McQueen, a 
sociologist. This article draws ideas 
from their presentations on generational 
differences in the two cohorts of stu- 
dents at the 1995 reunion of the classes 
for 1951-1953. The speeches constitute 
a penetrating qualitative study that dis- 
tills major socio-cultural themes emerg- 
ing in American society and their im- 



Figure 1 ■ Ronald Dear’s Reworked Social Welfare Agenda 

■ The national political agenda must again be infused with a concern for poverty, hun- 
ger, homelessness, unemployment, crime, and urban decay. 

■ Disparities in life quality between the more and less affluent must not be ignored. 

■ Social programs must be updated and modified based on evaluations of their effec- 
tiveness, and new initiatives tested. 

■ Program changes should be made incrementally rather than through radical surgery. 

■ The tax system must be made more progressive. 

■ Defense expenditures need to be reduced. 

■ The level of Social Security benefits may need to be more consistent with the level of 
contributions to the system. 

■ The U.S. needs to develop an annual or biennial broad-based “social indicators” re- 
port. 

■ The U.S. should recognize and capitalize on its diversity without allowing special inter- 
ests to fragment the society’s consensus on social welfare issues. 

■ Social policy reforms need to give more attention to basic social class divisions, em- 
phasizing solutions to systemic problems. 

■ Social policy must give more attention to increasing collective life quality, rather than 
life quality only for certain groups. 
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pact on the values, attitudes and be- 
havior of two young adult generations 
separated by a span of forty-five years. 
These were the critical years following 
World War II which witnessed rapid 
and far-reaching economic, social and 
cultural change in American society. As 
a valuable illustrative case, these com- 
mentaries increase our understanding 
of the base of societal leadership that 
helped form the social policies of the 
past three decades, and will shape its 
immediate future. 

The Analysts 

Majoring in psychology, Albert Mc- 
Queen was one of a small number of 
African Americans graduating from 
Oberlin in 1952. Returning to teach at 
Oberlin in 1966, he became a member 
of the joint sociology/anthropology fac- 
ulty and served as its chair for an ex- 
tensive period prior to his retirement in 
1995. He was one of the earliest black 
faculty members at Oberlin, Oberlin 
having been the first liberal arts college 
to admit African Americans and women 
on a regular basis. At the University of 
Michigan in the early sixties, McQueen 
received a Ford Foundation Fellowship 
to pursue African Studies at the Univer- 
sity of London, and a Research Fellow- 
ship at the University of Ibadan in Ni- 
geria to conduct research on selected 
youth problems in Nigeria and Kenya. 

A grant from the Social Science Re- 
search Council supported the analysis 
of his cross-cultural data. McQueen has 
served as President of the Association 
of Black Sociologists, and has held a 
number of positions in the American 
Sociological Association. He has a con- 
tinuing interest in the processes in- 
volved in youth socialization and their 
incorporation within the social struc- 
tures of different societies and historical 
periods, with a particular interest in mi- 
nority youth. 

Nancy Dye majored in history at Vas- 
sar College and received a doctorate in 
history from the University of Wisconsin 
at Madison. She was a history professor 
at the University of Kentucky and Dean 
of Arts and Sciences at Vassar College 
before becoming Oberlin’s thirteenth 
and first woman president. Dye re- 
ceived the Outstanding Professor Award 
from the University of Kentucky, as 
well as fellowships from the Rockefeller 
and Ford Foundations, and a grant from 
the National Endowment for the Hu- 
manities. She has served on numerous 
education-related boards, including the 



editorial board of the Journal of Ameri- 
can History. She has authored a book 
on feminism and the labor movement, 
co-edited a book on gender, class and 
ethnic issues, and written a number of 
book chapters, articles and papers on 
women's history topics, including is- 
sues dealing with work, health, race, 
class and femininism. 

The following overview is based on 
Dye’s and McQueen’s presentations at 
a 1995 reunion program titled “Under- 
standing Student Generations in a 
Changing World.” 

Professor McQueen’s Analysis 

McQueen focuses his analysis on 
three sets of developmental tasks that 
pose the greatest challenges to college 
students, and the historical contexts 
that shape youth’s perspectives and life 
options. The three most important sets 
of developmental tasks facing students 
are these: 

1. Developing and consolidating a satis- 
factory sense of identity and com- 
mitments to a set of values and goals 
for one’s life. 

2. Pursuing a liberal education: 

■ To acquire knowledge, skills and 
orientations that expand one’s in- 
tellectual horizons to include 
many fields of interest. 

■ To learn to think independently 
and critically. 

■ To creatively integrate knowledge 
from disparate fields. 

■ To gain specialized knowledge 
and skills useful in making career 
choices. 

■ To become sensitized to the moral 
and ethical aspects of knowledge 
and its uses as a basis for enlight- 
ened citizenship. 

3. Developing self-confidence and a 
sense of belonging, by being recog- 
nized and respected as a valued per- 
son by significant individuals and 
groups. 

McQueen sees the roots of early 
identity and commitments in one’s so- 
cialization prior to young adulthood — 
in family, peers, the educational sys- 
tem, churches, and the immediate and 
larger communities. College, he says, 
serves to concentrate and speed the ac- 
quisition of self-knowledge and a 
knowledge of the worlds of which one 
is an integral part. His view is that 




unique societal influences at particular 
historical moments can have a decisive 
impact on the efforts of young people 
to cope with developmental. tasks and 
on the results of such efforts. For com- 
parative purposes, McQueen concen- 
trates on two cohorts of students during 
the distinctly different historical periods 
of the late 1940s to early 1950s, and the 
late 1980s to early 1990s. 

Avoiding Overgeneralizations 
About Intergenerational 
Differences 

As a researcher, McQueen alerts us 
to the dangers of stereotyping college 
student cohorts as “conformist and si- 
lent’’ (the phrase often used to de- 
scribe the generation of the fifties), as 
“rebellious” (the sixties’ epithet), and 
as “conservative and materialistic” (the 
frequent assessment of the 80s’ genera- 
tion). Although these kinds of labels 
may contain important elements of 
truth, he cautions his audience not to 
become entrapped in such broad gener- 
alizations. 

McQueen points to Paul Loeb’s stud- 
ies of American college students be- 
tween 1987 and 1993 reported in Gener- 
ations at the Crossroads. Loeb found 
that the majority of the students in 
these studies, who were confronting 
personal insecurities rooted in precari- 
ous socio-economic conditions and un- 
certain futures, tended to be politically 
withdrawn and somewhat apathetic. 
Consequently he labeled them adapt- 
ers. Loeb also identified a significant 
minority of students as activists. These 
were the students who took public 
stands and engaged in actions intended 
to bring about desired social changes. 

The Influence of Societal Change 

McQueen highlights certain key soci- 
etal changes in American society fol- 
lowing World War II that have influ- 
enced the ways in which college 
students have dealt with essential de- 
velopmental rites of passage: 

■ The economic boom of the late 

1940s and early 1950s. 

■ Expanded job opportunities and in- 
creased socio-economic upward mo- 
bility. 

■ Suburbanization and the growth of 

the middle class. 

■ Increased geographic mobility via ex- 
panded transportation systems. 
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■ Expanded communications, with the 
development of electronic informa- 
tion production technologies. 

McQueen points out that these changes 
in the decade after World War II did 
not mirror substantial changes in the 
basic underpinnings of the society — 
that is, in traditional family and com- 
munity values, attitudes and behavior. 
He reminds us, however, that deeply- 
rooted, oppressive prejudices and pat- 
terns of discrimination persisted, and 
that the era of the Cold War and the 
Korean War produced “highly virulent 
pressures for ideological conformity.” 

In this era there was minimal ambi- 
guity at Oberlin about what constituted 
appropriate academic standards and in- 
dividual behavior. The Oberlin heritage, 
built on Christian doctrines and values, 
has at its core principles of socio-politi- 
cal and cultural progressivism. The au- 
thority of the administration and faculty 
as carriers of this heritage was rarely 
tested. Nevertheless there was diversity 
within the student body. Many could 
be classified as adapters , a large pro- 
portion as activists , and a small number 
as rebels (who, for example, struggled 
for and accomplished the establishment 
of radical new living arrangements in 
student cooperatives.) 

The late 1980s and early 1990s was a 
time of dramatic changes that contrast- 
ed sharply with those of the earlier pe- 
riod. Among the forces McQueen be- 
lieves have played a large part in 
shaping the life conditions, perspec- 
tives and future prospects of college 
students of this era are the following: 

■ Economic and social stresses result- 
ing from a widening gap between 
the rich and the poor; growing glo- 
bal interdependence and competi- 
tion; increasing federal deficits and 
policy changes; increasing costs of 
educational and medical services; 
and continuing degradation of the 
environment. 

■ Vast expansion of ever-changing me- 
dia-promoted consumption patterns 
and lifestyles. 

■ Significant erosion of many tradition- 
al family and community values and 
arrangements. 

■ Growth of major social problems 
such as homelessness, drug use and 
abuse, ethnic and gender conflict, 
violence, and AIDS. 

■ Dramatic increases in the production 
and near-instant dissemination of in- 



formation of every kind via constant- 
ly developing electronic technologies. 

Despite indications of social progress, 
McQueen believes these forces have 
created substantial anxiety and uncer- 
tainty for college students as they seek 
resolutions to essential developmental 
tasks. Added to this is the fact that the 
student movements of the 1960s and 
1970s undermined the guiding and 
mentoring roles of many adult authori- 
ties (including parents), leaving an ab- 
sence of significant guide-posts in the 
lives of many young people, who chose 
independence and self-regulation as 
they contended with the challenges of 
growth and development. 

One expression of this new indepen- 
dence is what McQueen calls “the pow- 
erful identity politics” of the many spe- 
cial interest groups that have emerged 
on college campuses. In contrast to the 
earlier emphasis on “melting pot” ideas, 
these groups are inclined to concentrate 
on the pursuit of their own interests 
and rights, which they may define in 
terms of political, ethnic, racial, reli- 
gious, cultural or other characteristics. 
This has led to pervasive fragmentation 
in campus life. While such widespread 
preoccupations with special interest 
group goals surely yield some positive 
benefits for those involved, McQueen 
feels that the general trade-off has been 
“deep uncertainties and insecurities 
among college students, widespread 
conflicts in student life, and an atten- 
dant loss of a sense of community.” 
However, he sees important counter 
tendencies, in that many students who 
are involved in identity politics also seek 
to cultivate common interests on cam- 
pus and to serve the larger community 
for the common good. 

President Dye’s Analysis 

Dye proposes that the characteristics 
of Oberlin students in 1995 are not dis- 
similar to those of previous eras in these 
respects: 

■ A passionate interest in the world. 

■ The desire to explore this world intel- 
lectually, socially and aesthetically. 

■ A strong commitment to achieve at a 

high academic level. 

■ A deep commitment to social 

progress. 

However, she also sees major differenc- 
es between current students and those 
of previous eras. She illustrates some of 
these with a letter from a fifties’ alum- 



nus and professor of history, Geoffrey 
Blodgett: 

We had the enormous good luck to 
believe that we possessed a desir- 
able future. The society we were 
getting ready to enter seemed, on 
balance, to be a good society. Most 
of us were eager to find our places 
in its structure and fulfill ourselves 
in ways expected of us. We were 
remarkably career-oriented and 
achievement-oriented. We thought 
we were needed by society, and by 
the professions we wanted to pur- 
sue. The jobs were out there wait- 
ing for us. We accepted the ethic of 
apprenticeship in order to get 
ready for them. We entered train- 
ing programs and graduate 
schools to find out what was ex- 
pected of us, required of us, and 
we did it. 

This kind of attitude, Dye says, does 
not describe the world view of college 
students in the ’90s. 

Influences Shaping the 
Perspectives of Youth in the 
1990s 

Dye emphasizes several factors in 
seeking to explain some of the differ- 
ences in orientation between post- 
World War II students and students to- 
day: 

Differences in Cultural 
Realities 

A common “normative culture,” Dye 
says, characterized the 1940s and 
1950s, pulling the society together 
around a singular set of values, mores 
and attitudes about the privileges and 
responsibilities of citizenship, which 
was associated with a patriotic nation- 
alism. A relatively common view of 
country and society supported the cen- 
tral role of political institutions and citi- 
zens’ acceptance of them — an era she 
titles “middle class nationalism.” This 
middle class, rising out of the prosperi- 
ty in the trade and manufacturing sec- 
tors, represented a strong nationalistic 
base that at the same time provided a 
“sense of place” and “historical conti- 
nuity.” It helped prevent a deterioration 
of American society into contending 
factions, and promoted a common feel- 
ing of community attachment and obli- 
gation. 

The society of the 1990s, she con- 
tends, is more difficult to understand, 
to enter, and to accept. Traditional con- 
cepts of nation and citizenship, and of 
institutional authority, are being ques- 
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tioned. Diversity is challenging our un- 
derstanding of community also. 

Breakdown of the Nation-State 

In the post- World War II years, 
young people grew up with a loyalty to 
country and its institutions, and a de- 
sire to work for change within that sys- 
tem. Again quoting from Blodgett’s let- 
ter: 

In courses in American history and 
foreign policy, we began to absorb 
the new realist concepts of national 
self-interest and controlled conflict 
which underwrote the Cold War 
containment policy and displaced 
older Wilsonian concepts of inter- 
national law and moral idealism as 
proper guides to foreign affairs. 

... In the arena of domestic Ameri- 
can history, “consensus” gradually 
replaced “conflict" as the key 
phrase for explaining how Ameri- 
cans normally thought and be- 
haved toward one another. As we 
learned to locate the terms of 
American consensus in the past, 
more often than not we also 
learned to identify those terms as 
values of our own. From political 
science we absorbed comparable 
lessons. We discovered the intrica- 
cies of the peculiar American party 
system. . . . We learned how the 
system worked, for all its idiosyn- 
cracies, to master the tensions of a 
pluralistic society. Implicitly, and 
sometimes explicitly, we received 
the message that it was better to 
join a party and work within it for 
improvement than to stand on the 
periphery wringing our hands in 
dismay.. .We learned to our sur- 
prise that most professors of politi- 
cal science frowned on political in- 
dependents as soft-minded, 
idealistic troublemakers. The inde- 
pendents’ notion that party politics 
spelled a degradation of pure rep- 
resentative democracy was, we 
were told, a self-fulfilling prophe- 
cy, because it removed them from 
effective involvement in the system 
they disliked. 

Dye proposes that rising economic 
internationalism and the decline of the 
importance of the nation-state have re- 
duced students’ national loyalties, their 
view of the responsibilities of citizen- 
ship, and their interest in social activ- 
ism within the political system. In a 
1994 survey of incoming Oberlin fresh- 
man, only 31.9% said that “keeping up 
with political affairs” was an important 



goal — the lowest percentage in the 
history of the survey. The percentage 
who said they discuss politics frequent- 
ly also reached its lowest point. Dye 
surmises that specialized identity 
groups have replaced some of the 
broader political commitment that was 
such a strong quality of Oberlin stu- 
dents in the past. 

As a result of these and associated 
trends, such as increasing economic in- 
equality, Dye feels that college students 
today, and to large extent young adults 
generally, lack a belief that they will 
have the future they desire. They do 
not feel assured that they will be “bet- 
ter off” than their parents. They are the 
first generation to harbor that anxiety. 

In fact, many expect to be “worse off” 
in terms of standard of living, with a 
new agnosticism about the future. It 
may be secure, it may not be. They 
may marry and have children, they may 
not do so. They may have traditional 
sexual orientations, they may not. They 
feel more ambiguity and uncertainty 
than before. 

Changing Conceptions 
of the Family 

Close to 35% of Oberlin students 
now come from single-parent house- 
holds. Many come from blended fami- 
lies. Dye suggests that these students 
view relationships of all kinds as far 
more fragile and fluid than was the 
case in earlier eras. 

Some Comments 

The analyses of disparate college 
generations tell us important things 
about the differences emerging in so- 
cialization patterns and attitudes relat- 
ed to significant change in American 
society. Increasing uncertainty about 
personal life choices and anxiety about 
one’s economic future are pervasive in 
the current youth generation, in the 
context of weakening ties with institu- 
tional authority and the political sys- 
tem. Clearly, these changing personal 
perspectives and world views have im- 
plications for youth education, training 
and employment policy. A prominent 
implication is that such policy must be- 
gin with a broader understanding of 
the relationship between societal and 
global economic, social and cultural 
change and the achievement of generic 
youth development tasks. 



Editor’s Note 



A November 30, 1995 article in the 
New York Times, “Boomers: ‘Not As I 
Did’ Parents,” provides interesting an- 
ecdotal support for one of the themes 
in the Oberlin presentations, namely 
the legacy of too early self-indepen- 
dence in a more complex society and 
world. The journalist, Trip Gabriel, 
claims that surveys suggest that the 
Baby Boom generation (now age 31- 
49) has a strong desire to protect chil- 
dren from a world that has grown more 
threatening, and from an era in which 
they believe they were afforded too 
much freedom too soon. These con- 
cerns about the future of children, 
about anomie and cynicism, cuts 
across political and racial lines, he 
says. Gabriel quotes from Generations, 
co-authored by historian Neil Howe, to 
make his point: 

What Boomers do not want is for 
their kids to grow up like Generation X 
did — no order, no structure, no inno- 
cence. Facing too many choices. Being 
told the answers to questions they nev- 
er had, at too early an age. Exposed to 
too much. 

Gabriel claims that the mantra of the 
Boomers was “question authority.” Now 
they are the authority. Surveys reveal a 
new more restrictive, rulemaking child 
rearing movement among them. A 
1995 Roper survey found that good 
manners and politeness were at the 
top of the list. Responsibility was sec- 
ond. Independence and imagination , 
high priority in their own era, were 
much lower on the list. 

But some experts see this new inter- 
est in placing limitations on behavior, 
and instilling values such as teamwork, 
as a product of both parents working 
and having less time to serve as role 
models for their children. Many mea- 
sures of adolescent well-being, for ex- 
ample, are declining: increasing mari- 
juana and LSD use, a rising incidence 
of sexual intercourse by age 15, a 
greater potential for acquiring sexually 
transmitted diseases, increasing arrest 
rates for violent crime, a growing media 
focus on sex and violence. Some youth 
specialists do not see a turnaround, 
Gabriel says, until the larger second 
wave of children of the Boomers (now 
8 and under) enter adolescence. 
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Interested readers may want to delve 

into the following: 

Dye, N. (co-editor with Noralee Frankel) 
Gender, Class, Race and Reform in the 
Progressive Era. Louisville, KY: Univer- 
sity of Kentucky Press, 1991. 

Dye, N. “The Rise of the New Woman: 
Women’s Work, Women's Politics, 1870- 
1928," in M. Cantor (editor) Main Prob- 
lems in American History, Volume 2. New 
York, NY: Dorsey, 1988. 

Dye, N. As Equals and As Sisters: Femi- 
nism, the Labor Movement, and the 
Women's Trade Union League of New 
York. Columbia, MO: University of Mis- 
souri Press, 1981. 

Horowitz, H. L. Campus Life: Undergraduate 
Cultures From the End of the 18th Cen- 
tury to the Present. Chicago, IL: Univer- 
sity of Chicago Press, 1987. 

Littwin, S. The Postponed Generation: Why 
American Youth Are Growing Up Later. 
New York, NY: William Morrow and Co. 
1986. 

Loeb, P. R. Generations at the Crossroads: 
Apathy and Action on the American 
Campus. Berkeley, CA: University of 
California Press, 1989. 

Modell, J. Into One’s Own: From Youth to 
Adulthood in the United States, 1920- 
1975. Berkeley, CA: University of Cali- 
fornia Press, 1989. 



Rising Income 
Inequality and 
Poverty 

Sheldon Danziger and Peter 
Gottschalk have collaborated on numer- 
ous studies of poverty and unemploy- 
ment, and have jointly published their 
results in article and book form. This 
article reviews their most recent, very 
timely book, America Unequal , pub- 
lished by the Russell Sage Foundation 
and Harvard University Press in 1995. 
Danziger is Professor of Social Work and 
Public Policy at the University of Michi- 
gan. Gottschalk is Professor of Econom- 
ics at Boston College. 

The theme of the book is that the 
global and domestic economy, techno- 
logical change, family structure and 
public policies have contributed to the 
gradual decline in family incomes, to 
rising economic inequality, and to an 
erosion of job opportunities for lower- 
skilled workers. Based on empirical 
studies made possible by grants from 
the Russell Sage Foundation and the 
Presidential Initiatives Fund of the Uni- 
versity of Michigan, the authors reject 
the notion that increasing poverty in 
the U.S. is the result of a weakening 
work ethic, a legacy of social programs, 
or associated with the government’s 
failure to shore up the middle class and 
address the increase in poverty. Their 
recommendations fall heavily on de- 
mand-side as opposed to supply-side 
solutions. 

A Departure from the Past 

The authors comment that the ine- 
qualities in the America of the 1990s 
represent a substantial departure from 
our recent economic history. Family in- 
comes doubled in the two decades fol- 
lowing World War II. They have grown 
only minimally since then. More strik- 
ing, many workers have lower real 
earnings than in the 1970s. Youth and 
young adults have assumed that they 
would experience the same mobility 
and affluence as their parents, but the 
authors claim this hope has dimmed in 
the new economic and technological 
environment. 

The authors note that in the era of 
the 1950s and 1960s, most families 
were living out the “American 
dream” — good wages and fringe bene- 
fits, house ownership, savings. The in- 
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come of all American social classes rose 
at nearly the same rate. The concept of 
“a rising tide lifting all boats” seemed 
to reflect real life, and indeed the pov- 
erty rate decreased, by half between 
the late 1940s and early 1960s, and 
again by half by the early 1970s. Now 
wages are much lower and fewer jobs 
offer fringe benefits. Unemployment 
rates for both white-collar and blue-col- 
lar workers are higher. Both adults in 
two-parent families must work to allow 
for continuing economic support and 
home ownership. The retirement age 
has been raised, and social security 
benefits in the future will mirror the 
lower rate of return on social security 
taxes, significantly affecting young 
workers. 

These younger workers face a soci- 
ety characterized by rising inequalities 
in wages, income and wealth, and a 
high rate of poverty, according to the 
authors. Such conditions are the legacy 
of the 1980s and early 1990s, they say, 
during which growth in income was 
skewed toward the top of the income 
distribution, and inequalities within 
and between demographic groups wid- 
ened. Different groups began to experi- 
ence disparate economic fortunes. The 
slender gains went disproportionately to 
the old, the better educated, two-per- 
son families, and whites. 

The Inner-City Syndrome 

The popular view has been that 
these changed conditions reveal a lack 
of work effort and skills. The public 
looks to inner city poverty, single moth- 
ers, and displaced blue collar workers 
as the primary victims. Danziger and 
Gottschalk say this is a distortion of re- 
ality. The new inequalities have affect- 
ed all population groups, even college 
graduates. In 1991, for example, 16% 
of male graduates and 26% of female 
graduates between age 25 and 34 
worked at some time during the year 
but earned less than the poverty line 
for a family of four. In 1973, this was 
true for only 11% of the men and 37% 
of the women. The demand for less- 
skilled, less-experienced workers has 
decreased noticeably. It is gradual, 
even though not visibly dramatic 
changes in the economic environment, 
the authors contend, that explain the 
changes in Americans’ standard of liv- 
ing, not changes in the American work- 
er. And the origins of the problem lie in 
the early 1970s. It is these structural 
economic changes that are the authors’ 
primary variables in the search for ex- 
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planations for rising inequality and pov- 
erty. 

The recovery in the 1980s looked like 
a period of renewed progress, but the 
data hardly support this conclusion. 

This era did lead to job growth, declin- 
ing unemployment and inflation, and 
an increase in the gross national prod- 
uct — but only a small portion of the 
American population benefited. This 
period “set the stage,” the authors say, 
“for both a change in academic think- 
ing about the level of distribution of liv- 
ing standards and a change in presi- 
dential administrations. The issue of 
fairness moved from the background to 
center stage in the early 1990s.” Re- 
search on inequality and poverty began 
to attract more attention. 

A “revisionist” view of the 1980s 
emerged in the context of low wages 
and benefits, worker displacement and 
dislocation, and continued high unem- 
ployment rates for certain groups. And 
the media began to notice the problems 
associated with corporate restructuring 
and downsizing, such as the unemploy- 
ment of young college graduates, col- 
lege-educated middle managers, and 
highly-paid blue collar workers. Consis- 
tent with this concern, the authors 
show that since the recession of 1973 
to 1975 poverty rates rose more during 
each recession than following each re- 
covery, parallel with a declining general 
standard of living and lowered consum- 
er confidence. Poverty rates began to 
move in a different direction than eco- 
nomic growth. During the 1980s the top 
end of the income distribution benefit- 
ed much more than others. Their earn- 
ings and property income went up and 
their federal tax rates went down. 

As data revealed an increasing gap 
between rich and poor, public policy- 
makers could hardly avoid acknowledg- 
ing the effects of the internationaliza- 
tion of markets, technological 
breakthroughs and structural economic 
change, and began to seek solutions. 

But proposing solutions was not easy. 
Macro-economic approaches have been 
one focus, tax and social policies anoth- 
er. To assist the reader in evaluating the 
alternatives, Danziger and Gottschalk 
review the evidence and pose these 
questions: 

■ Why have poverty rates remained so 

high? 

■ Why has the gap between the rich 

and poor, and between the rich and 

the middle class grown so wide? 

O 



■ Why didn't the gains of the 1980s 
“trickle down” more to the middle 
class and the disadvantaged? 

■ What changes have made more of 
the population rich without reducing 
poverty? 

■ What can be done to reduce poverty 
and the economic inequalities that 
have widened over the past two de- 
cades. 

Answering the Questions 

The authors suggest that most re- 
searchers did not see coming the 
change in average earnings. Neither did 
they envision the increase in earnings 
inequality, the continuing high rates of 
poverty, or the impact of the weakening 
of governmental concern and attention 
to poverty and inequality in the 1980s. 

Family Income 

The authors point out that both me- 
dian family income, the most common 
measure of standard of living, as well 
as mean family income grew almost 
continuously from 1949 through 1973. 
The public perception in the early 
1970s was that the U.S. was much rich- 
er than earlier, and more affluent than 
other industrialized nations. Neither of 
these visions accurately reflect current 
reality. 

The 1973-1975 recession was the 
longest and steepest since the late 
1930s. In 1980, the country was again 
in recession, followed by a brief recov- 
ery. At this point median family income 
was 7.4% below its 1979 value and 4% 
below the 1973 level. For the first time 
since the Great Depression, the " typical ” 
family was worse off than it was a de- 
cade earlier. The recovery beginning in 
1982 was relatively strong and endur- 
ing. Mean and median income grew. By 
1989 it finally rose above the previous 
high in 1979. But economic inequality 
and poverty kept increasing. The 1970s 
and 1 980s had essentially produced little 
improvement in the median standard of 
living. Adjustments for the shift toward 
more female-headed families failed to 
explain this paradox. 

What stands out in the authors' 
analysis of family income data is that 
all of the increased inequality between 
1982 and 1989 is explainable in terms 
of the income gains for the richest 5% 
of American families. This top group re- 
ceived more total income than the en- 
tire bottom 40%. The middle class was 



not exempt from this change. In the 
late 1980s and early 1990s, the share of 
income held by the middle class was 
lower than in any other period since 
World War II. Much of the growth in 
the incomes of married-couple families 
was due to the wages and work effort 
of wives, not to the rising wages of 
husbands. Meanwhile, the tax cuts of 
1981 benefited the highest income fam- 
ilies, not the middle class. In fact, Dan- 
ziger and Gottschalk see the 1980s as 
an “historical anomaly,'' an era of rising 
family incomes and rising inequality. 

The continued and unprecedented in- 
crease in inequality during the recovery 
of the 1980s challenged the convention- 
al wisdom about the effects of econom- 
ic cycles. 

The Rich and the Poor 

In their analyses of wealth and pov- 
erty, the authors chose a conservative 
measure of poverty by using the official 
poverty line. In 1991 this meant $13,924 
for a family of four. Since no compara- 
ble official definition of wealth existed, 
they collected data on the top 5% and 
the top quintile of families. Their ap- 
proach was this: if the richest fifth of 
families is considered “rich,” and if 
their incomes remain the same while 
the other four-fifths fall, the share of in- 
come possessed by the richest fifth will 
rise, resulting in an increase in inequal- 
ity but not in .the proportion of families 
who are rich. That is, the level and dis- 
tribution of income influence the pro- 
portion of the population that is rich, 
but this segment does not increase un- 
less the absolute real incomes of the 
rich increase. 

For example, if every family's in- 
come doubled, there would be no 
change in income inequality, but 
poverty would decline and the 
percentage who were rich would 
increase. If, in contrast, the in- 
comes of most families stayed 
constant but the incomes of those 
near the top of the distribution 
increased, there would be no 
change in poverty, but inequality 
and the percentage of rich would 
increase. 

The poverty rate fell substantially 
during the rapid growth of the postwar 
years -- from approximately 40% of all 
persons in 1949, to 22% in 1959, to 
14% in 1969. In that same period, the 
rich increased from less than 1 % to 
more than 5%. The middle class also 
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grew rapidly, from 60% of all persons 
in 1949 to 80% in 1969. But although 
the poverty rate tended to rise and fall 
with the condition of the market, it be- 
gan to increase more during the reces- 
sions and to decline less during the re- 
coveries. Danziger and Gottschalk do 
not imply that economic growth no 
longer mattered, but rather that the re- 
lationship between the poverty rate 
and economic forces was more complex 
than thought. 

If the economy were to stay on its 
present course, the authors estimate 
that median family income would have 
to grow to $39,000 (over 13%) to return 
to the 1973 poverty rate level. Their 
point is that without dramatic changes 
in underlying demographic and eco- 
nomic factors, poverty is destined to 
stay well above the 1973 level. To sup- 
port this conclusion, the authors calcu- 
late the poverty rate using alternative 
measures. These lead to the same end 
result, or to an even higher poverty 
rate. 

At the beginning of the War on Pov- 
erty, the reasonable assumption was 
that poverty could be reduced by 
healthy economic growth, through mac- 
roeconomic control of the business cy- 
cle. The Reagan Administration, the au- 
thors remind us, blamed the stagnation 
of the 1970s on the rise in oil prices 
and on President Carter’s economic 
management, and they assumed that a 
speed-up in the economy would be 
shared equally by all groups. None of 
these assumptions was correct. 

Disadvantaged Groups 

A major change since World War II 
is the shrinkage of that part of the pop- 
ulation, in each decade, that has lived 
in families headed by the nonelderly 
and the non-Hispanic white. Danziger 
and Gottschalk caution us to review 
this trend in the context of the increas- 
ing proportion of the population that 
represents groups with lower-than-aver- 
age living standards and higher-than- 
average poverty rates — i.e. minorities 
and female-headed families. They say 
the data indicate that increasing hard- 
ship in these groups cannot be attribut- 
ed to “moral decline in the American 
family” or the relative increase in the 
size of the minority population. In the 
absence of demographic change, the 
economy still would have created such 
inequalities, and the hardships would 
have been more severe if women had 
not gone to work in unprecedented 
numbers. Furthermore, government pro- 
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grams have provided less cash assis- 
tance to low-income families in a period 
in which the economy has increased 
the need for such assistance. 

The authors’ research confirms that 
the past twenty years have witnessed a 
divergence of living standards among 
various groups in American society, 
with some of those receiving the lowest 
incomes experiencing the most stag- 
nant rise in living standards and the 
largest increases in poverty. Non-His- 
panic whites are better off than blacks, 
Hispanics and other minorities. Noneld- 
erly adults are in a better position than 
female-headed families. Workers with 
more education are faring better than 
those with less. Current poverty rates for 
minorities and persons in families head- 
ed by unmarried women are above 25%. 
Black , Hispanic and white children liv- 
ing in mother-only families have poverty 
rates as high as 40%. 

Children of Minorities 

The trends for minority children are 
particularly worrisome. While children 
benefited from the economic progress 
following World War II, they did less 
well in the 1970s and 1980s. Significant 
to the economic status of minority chil- 
dren, over the past two decades these 
children have been more likely to live 
with one parent than with two. Follow- 
ing 1973, while the median income for 
all African Americans increased by 
16.4%, the median income for black 
children did not increase, and the me- 
dian income for Hispanic children fell 
by 11%. This greater divergence for 
children than for other persons is of 
great policy concern. In 1991 , the medi- 
an standard of living for white children 
was 96% of all persons; for black and 
Hispanic children it was 40% and 44% 
respectively. In 1991 , the median black 
or Hispanic child lived in a family 
whose living standard approximated the 
standard for all Americans in 1 949 , only 
1.24 times the poverty line. 

Men's Incomes 

Over the past twenty years, only the 
elderly have experienced rising incomes 
and declines in poverty. This has been 
due mainly to government spending. At 
the same time the elderly’s safety net 
was improving, economic changes were 
unkind to working men. Men’s earnings 
increased substantially between the 
late 1940s and early 1970s, and then 
stopped — in fact, their real earnings 
were lower in 1991 than in 1973! In- 



creased work hours, particularly on the 
part of female spouses, has been the 
primary coping strategy. 

In 1949 men’s earnings represented 
82.7% of family income. Between 1949 
and 1969, growth in men’s earnings ac- 
counted for 73% of the rapid growth in 
family income, even though the propor- 
tion of married mothers with earnings 
doubled. In 1991, men’s earnings still 
accounted for 63.4% of total adjusted 
family income, the earnings of their 
wives representing 21.5%. Clearly the 
largest source of income for these fami- 
lies remained the men’s earnings. After 
the early 1970s, however, the growth 
rate of men’s earnings slowed, more 
wives entered the workforce, and wag- 
es and hours of work decreased. By 
1991, two-thirds of all mothers in two- 
parent families had earnings, but the 
absolute level of earnings of men had 
stagnated. 

Although men's earnings remained 
the largest source of family income , they 
accounted for only 2.5% of the growth 
in family income between 1973 and 
1991. In the 1990s wives' earnings actu- 
ally replaced those of men as the prima- 
ry reason for any growth in family in- 
come. Wives' earnings now accounted 
for 95 % of the increase in their chil- 
dren's standard of living. 

These results, based on careful sta- 
tistical studies, again convinced Dan- 
ziger and Gottschalk that it is adverse 
economic changes — in particular the 
declining earnings of men — that ex- 
plain current economic hardship, rather 
than sheer demographic change. Mean 
male earnings have hardly grown over 
the past two decades, and inequality in 
men’s earnings has increased. 

What the data show for white men 
holds true for minority men. The grow- 
ing inequality of the past two decades 
had similar effects on the living stan- 
dards of white and minority men. How- 
ever, since the median standard of liv- 
ing of minorities has been much lower 
than that of whites, the inequality is 
greater. The authors are firm in reject- 
ing explanations of the disparity in 
terms of "race-specific behavior.’’ They 
provide ample evidence that economic 
change is the primary culprit. 

A case in point is educational bene- 
fits. There has been a pattern of rising 
rates of low earnings for both white 
and black male high school graduates, 
with the two groups beginning to di- 
verge in the 1970s. This has been true 
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for the earnings of college graduates as 
well. The authors contend that conser- 
vative explanations of the differences in 
earnings between these two racial 
groups have vastly overestimated the 
role of individual motivation, family 
structure and detachment from the la- 
bor market. Although these factors 
have contributed to the disparity, the 
authors say, they do not explain much 
of the increase in economic hardship 
over the past twenty years. 

Female-Headed Families 

The public view has been that a 
large proportion of female heads of 
household are on welfare rather than in 
employment. But the authors’ data sug- 
gest otherwise. Women’s earnings in 
employment are the most important 
source of income for these families. 
Their earnings are low, however, be- 
cause women’s wage rates are lower, 
there are fewer adults to obtain em- 
ployment within the family, and these 
families receive so little cash assistance 
from government programs. Declining 
government cash transfers have not 
made much of a difference in their liv- 
ing standards. By 1991 government 
cash transfers amounted to only 10.3% 
of the mean adjusted income of fami- 
lies headed by women. They accounted 
for 20% in 1969. If these women were 
not working, the authors point out, 
their living standards would be consid- 
erably lower than they are now. In 
1973 cash transfers averaged .42 times 
the poverty line and accounted for 32% 
of the income of mother-only families . 

By 1 991 these transfers averaged only 
.24 times the poverty line , or 17% of to- 
tal adjusted family income. 

Increasing Poverty 

The authors give much of their at- 
tention to myths vs. mot causes. For ex- 
ample, they point to analysts who tend 
to blame the increasing divorce rate 
and rate of out-of-wedlock births for 
the high poverty rate, insisting that the 
data do not fully support this conclu- 
sion. Although the shift toward more 
female-headed families raised the pov- 
erty rate by 1.6% over an eighteen-year 
period, or by about .1% a year, they 
say, the poverty-reducing effect of the 
economic growth of the past two de- 
cades has been offset by the increased 
inequality of family income. The au- 
thors credit the much slower growth in 
living standards and the rising inequali- 



ty of incomes, particularly those of 
men, as the major explanatory influenc- 
es: 



There was much debate in the late 
1970s and early 1980s about whether 
increased inequality was correlated 
more with long-term structural changes 
in labor markets than with temporary 
responses to recessions. However, the 
authors point out that as economic 
growth expanded in the 1980s and 
earnings inequality also grew, it became 
obvious that the U.S. was experiencing 
long-term economic changes. Full-time, 
year-round workers, not only labor mar- 
ket groups with access to government 
transfers, were sustaining large increas- 
es in inequality. And earnings inequali- 
ty for all men continued to increase as 
reductions were made in unemploy- 
ment insurance and welfare. 

Technological as well as economic 
changes have played a significant role 
in increasing inequality, according to 
the authors. Computerization has in- 
creased the demand for higher-skilled 
workers with a resulting decrease in 
the employment opportunities and wag- 
es of those without computer skills. 

The de-industrialization of the U.S. has 
reduced earnings significantly, and up- 
ward earnings mobility has not 
changed appreciably over a long peri- 
od. 



Again, different experts have differ- 
ent explanations concerning the role of 
technological change, some criticizing 
the measures used to study its effects. 
But the authors point out that the nega- 
tive effects of such change on earnings 
has been well documented not only in 
general statistical studies but also nu- 
merous industry-specific studies. Also, 
it is now fairly well established that 
market forces that increase efficiency 
do so at the price of equity. Meanwhile, 
the public sector has been left to com- 
pensate for these negative effects of 
economic and technological progress. It 
chose not to rise to this occasion in the 
1980s. 

Job loss is sometimes given as an ex- 
planation for increased inequality. How- 
ever, the authors report that the extent 
of job loss was about the same in 1979 
and 1989, suggesting that this factor 
accounted for very little of the differ- 
ences in earnings. These losses mainly 
reflect changes in the industrial struc- 
ture of the labor force, according to 
Danziger and Gottschalk. For example, 



an increased number of workers are 
temporaries. The number of these 
workers grew sevenfold between 1972 
and 1991. By 1992 temporary employ- 
ment was as large as the steel and auto 
industries combined. And rather than 
responding to business cycles, the rate 
of temporary employment has shown a 
steady upward trend. 

The Author’s Conclusions 

The two researchers are quick to ac- 
knowledge that we are only beginning 
to understand the combination of com- 
plex influences that result in the 
growth of income inequality in the U.S. 
and the significant interactive effects of 
these factors. In summary, however, 
they find strong evidence regarding the 
role of these influences: 

■ Increased globalization of markets. 

a Improved efficiency, whose negative 

side is greater inequity. 

a Technological change. 

a Change in the structure of the labor 

force. 

These changes have both positive and 
negative effects on income inequality, 
but the overall trade-off has been in- 
creased poverty. Government’s primary 
obligation in a capitalist democracy, the 
authors suggest, is to develop policies 
that help offset the inequities resulting 
from these changes. 

An Anti-Poverty Strategy Based 
on the Research 

Danziger and Gottschalk are clear 
about the lessons we should have 
learned about public policy over the 
past several decades. Their message is 
that policies that are not consistent 
with the public’s perspectives and ex- 
pectations will not be successful or be 
appropriately implemented. For exam- 
ple, the negative income tax concept 
failed the public acceptance test. The 
potential efficiency in welfare mothers’ 
caring for their young children rather 
than working has failed also. Education 
and training programs have proven to 
be initially expensive, even though in 
some the costs have been offset by sig- 
nificant benefits to recipients. In terms 
of taxpayers’ tolerance for policies in- 
volving substantial spending, some ex- 
perts estimate it would take human cap- 
ital investments of about 1.7 trillion (in 
1986 dollars) to reduce inequalities suf- 
ficiently to bring the wage distribution 
up to the 1979 level! 
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Another lesson is that market forces 
alone are not capable of resolving the 
economic hardship we are now experi- 
encing, and therefore the authors view 
government action as essential. In that 
context, they concentrate on an ade- 
quate governmental response, particu- 
larly assistance in improving the em- 
ployment and earnings of those with 
low skills, and those most adversely af- 
fected by economic change: 

■ For those with jobs: expanded wage 
supplements and child care. 

■ For those who want to work but 
cannot find regular employment: 
transitional public service jobs at 
the minimum wage. 

■ Expansion of the Earned Income Tax 
Credit. 

■ Refundable income tax credits for 
the working poor. 

■ State income tax relief for the work- 
ing poor. 

■ Employer subsidies. 

■ Public service employment. 

■ Child support. 

The focus is on earnings supplements 
and work opportunities, rather than 
work incentives and requirements. The 
authors emphasize the demand side of 
the labor market, and apply their strat- 
egy to all those negatively impacted by 
economic, technological and related 
changes, not simply the poor, estimated 
to be roughly 47% of family heads. It is 
an impressive empirically-based and 
ambitious agenda in a period of uncer- 
tainty and unprecedented government 
retrenchment. 






Editor’s Note 



Those interested in other publica- 
tions of the authors may wish to delve 

into the following: 

Peter Gottschalk 

“Transfer Scenarios and Projections of Pov- 
erty into the 1 980s,” in Journal of Human 
Resources 16, 1981. 

Policy Changes and Growing Earnings Ine- 
quality in Seven Industrialized Nations. 
Research paper. Boston, MA: Boston 
College, Department of Economics, 

1994. 

[Co-authored with S. H. Danziger] “Macro- 
economic Conditions, Income Transfers, 
and the Trend in Poverty,” in The Social 
Contract Revisited: Aims and Outcomes 
of President Reagan’s Social Welfare 
Policy edited by Lee Bawden, 1984. 

[Co-authored with S. McLanahan and G. 
Sandefur] “The Dynamics of Intergener- 
ational Transmission of Poverty and Wel- 
fare Participation,” in Confronting Pover- 
ty: Prescriptions for Change edited by S. 
H. Danziger, G. D. Sandefur, and D. H. 
Weinberg, 1994. 

[Co-authored with G. Sandefur] The Growth 
of Earnings Instability in the U.S. Labor 
Market. Brookings Papers on Economic 
Activity 2, 1994. 



Sheldon H. Danziger 

[Co-authored with P. Gottschalk] ‘The Pov- 
erty of Losing Ground ,” in Challenge 
May-June, 1985. 

“The Poor,” in Human Capital and America’s 
Future edited by D. Hornbeck and L. 
Salamon. Baltimore, MD: Johns Hopkins 
University Press, 1991. 

[Co-authored with S. Danziger] “Child Pov- 
erty and Public Policy: Toward a Com- 
prehensive Antipoverty Agenda,” in 
Daedalus: America’s Childhood 122, 
1993. 

[Edited with P Gottschalk] Uneven Tides: 
Rising Inequality in America. New York, 
NY: Russell Sage Foundation, 1993. 

[Edited with G. Sandefur and D. Weinberg] 
Confronting Poverty: Prescriptions for 
Change. Cambridge, MA: Harvard Uni- 
versity Press, 1994. 



Readers are also encouraged to 
read the National Commission for Em- 
ployment Policy’s On Shaky Ground : 
Rising Fears About Incomes and Earn- 
ings by Steven Rose, published in 
1994. Rose’s analysis supports the the- 
sis in America Unequal. The conclu- 
sions in this monograph are these: 

■ The American economy is providing 
upward mobility for a declining pro- 
portion of Americans. 

■ Lack of access to post-secondary 
education is a barrier to job oppor- 
tunities. 

■ A widening income gap is generat- 
ing anxiety even among well-edu- 
cated and experienced workers. 

■ A declining number of men are up- 
wardly mobile — in every income, 
age and education category. 

■ Average male earnings over the en- 
tire decade of the 1980s were 4% 
lower than the 10-year average of 
the 1970s. 

■ Women are earning more than men, 
due to longer work hours. 

■ Higher family living standards are 
depending on women’s earnings. 

■ Young adults are much less suc- 
cessful now in advancing their ca- 
reers. 

■ The relative economic standing of 
black men has fallen dramatically. 

The U.S. is viewed as being in the 
midst of a profound economic transi- 
tion. Those with access to skills and 
technologies are doing relatively well. 
But new skill and technical require- 
ments are polarizing the economic 
well-being of those with the necessary 
skills and those without. 

The author asks for more flexibility 
in the labor market, through portable 
health care and pensions, education 
and training, counseling and job assis- 
tance, accurate labor market informa- 
tion, family support, and partnerships 
between capital and labor. The push is 
for more adaptive labor market policies 
which help workers adjust to significant 
change. 



tion Forum ■ Issue 12 ■ Summer 1997 



13 





Features 



A Victim of Policy 
Neglect? Wealth 
Inequality 

Poverty specialists focus most often 
on social and income inequality. Wealth 
inequality is frequently ignored. How- 
ever, the latter issue is beginning to 
capture analysts’ attention in the 
1990s — the extent of current dispari- 
ties in wealth are unprecedented. The 
combined effects of income and wealth 
inequality have significant implications 
for American society, and therefore for 
its youth. 

Consistent with the increasing inter- 
est in wealth inequality, we review Ed- 
ward N. Wolff's new book, Top Heavy: 

A Study of the Increasing Inequality of 
Wealth in America , published in 1995 
by The Twentieth Century Fund Press. 
Wolff is professor of economics at New 
York University. He is also managing ed- 
itor of the Review of Income and 
Wealth, associate editor of the Journal 
of Population Economics and Structural 
Change , a council member of the Inter- 
national Association for Research on In- 
come and Wealth, and a member of the 
editorial board of Economics Systems 
Research . His principal areas of research 
are productivity growth and the distri- 
bution of income and wealth. 

A Perspective onWealth 
Inequality 

Wolff confirms that most discussions 
of distributional issues have concentrat- 
ed singularly on income, or the mea- 
sure of a household’s economic position 
from year to year. His position is that 
wealth is a better indicator of long-term 
economic security. Wealth is the sum of 
the current value of all the assets a 
household owns minus all the house- 
hold’s liabilities. Wealth, as well as in- 
come, can vary over time but the 
former, Wolff insists, is the foundation 
for a family’s long-run security. 

Although certain kinds of income 
come from wealth, such as interest on 
savings or mutual fund accounts or 
rent, Wolff establishes that income and 
wealth are not identical. Some kinds of 
income do not flow from household 
wealth and many forms of wealth do 
not produce income. Although both are 
sources of well-being, wealth is the 
critical resource. Families differing in 
their income and families differing in 



their level of wealth can have quite dif- 
ferent life quality. In periods of eco- 
nomic stress due to unemployment, ill- 
ness or family breakup, wealth is an 
important cushion. And fortunes can 
be a source of economic power and so- 
cial influence over a number of genera- 
tions. 

Contemporary Wealth Inequality 

The 1980s offered enormous benefits 
to the top of the wealth distribution, 
while the rest of the population experi- 
enced a decline. In fact, 80% of house- 
holds experienced a decline between 
1983 and 1989, if one looks exclusively 
at real financial wealth, such as bank 
accounts, and stocks and bonds. Even 
if housing wealth is added, the bottom 
40% of households had less wealth in 
1989 than in 1983. 

During the same general period, the 
racial distribution of wealth declined 
from an already low level. The relative 
income of African Americans, Wolff 
says, held steady at approximately 60% 
of white income in the 1980s, but the 
relative wealth of most black families 
deteriorated. Wolff sees historical differ- 
ences in wealth between whites and 
blacks as a legacy of slavery, discrimi- 
nation and low incomes — but this con- 
dition grew worse between 1983 and 
1989. In 1983, the median white family 
had eleven times the wealth of the me- 
dian nonwhite family. By 1989 this ratio 
had risen to twenty times. Now more 
than one in three nonwhite households 
have no positive wealth at all, com- 
pared with one in eight white house- 
holds. 

HistoricalTrends inWealth 
Inequality 

The 60-year high in wealth inequali- 
ty in the U.S. was 1989, with the top 
1% of wealth holders controlling 39% 
of total household wealth. The richest 
1% owned 48% of the total. However, 
the rise in wealth inequality from 1983- 
1989 was unusually dramatic. The 
share of the top 1 % of wealth holders 
rose by 5%. The wealth of the bottom 
40% declined. Almost all of the abso- 
lute gains in real wealth were absorbed 
by the top 20% of wealth holders. 

American real wealth grew between 
1962 and 1983, growing even faster af- 
ter that. American marketable wealth 
actually grew twice as fast between 
1983 and 1989 as it did between 1962 
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and 1983. It had almost doubled since 
1962. Average financial wealth grew 
faster than marketable wealth during 
1983-1989, reversing the relationship of 
the 1962-1983 period. Average house- 
hold income also grew faster in 1983— 
1989 than in 1962-1983. But in both pe- 
riods average income grew more slowly 
than average wealth. The growth in av- 
erage wealth tended to mask changes 
in its distribution. Wolff says we must 
look at medians rather than means. The 
median of the wealth distribution is de- 
fined as the level of wealth that divides 
the population of households into two 
equal-sized groups. If only the top quin- 
tile experiences an increase in growth, 
median wealth is unaffected. When 
trends in the mean deviate from trends 
in the median, this tells us that gains 
and losses are unevenly distributed. 

During 1983-1989, mean wealth in- 
creased by 23% but median wealth 
grew only by 8%, with the bulk of the 
gains concentrated at the top of the 
distribution. The share of marketable 
net worth held by the top 1% rose to 
39% in 1989, while the share held by 
the bottom 80% fell by more than one- 
fifth. This trend is reflected in financial 
net worth , which was distributed even 
more unequally than total household 
wealth. 

In 1989, the top 1% of financially 
wealthy families owned 48% of the to- 
tal. The top one-fifth accounted for 
94% of total financial wealth, and the 
second one-fifth accounted for nearly 
all of the remainder. And the concen- 
tration of financial wealth increased to 
the same degree as marketable wealth. 
Furthermore, the top 1% of wealth 
holders received 62% of the total gain 
in marketable wealth between 1983 
and 1989. The bottom 80% received 
only 1%. This trend was a shocking 
turnaround, Wolff says, from the 1962- 
1983 period, when every group enjoyed 
some share of the overall growth in 
wealth, and gains were roughly in pro- 
portion to the share of wealth held in 
1962. 

Income distribution also became 
more concentrated between 1983 and 
1989. Most of the relative income gain 
went to the top 1%, whose share of to- 
tal household income grew by 4%. Al- 
most all the loss in income was felt by 
the bottom 80% of the income distribu- 
tion. 
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Changes in the Structure of 
HouseholdWealth 

Wolff focuses his analysis of struc- 
ture on demographic differences, and 
differences in the composition of 
wealth. Households in the age group 
45-69 are now the wealthiest group in 
the U.S., with those 70 and over the 
next wealthiest, and households under 
45 the next. The average wealth of 
white and nonwhite households contin- 
ued to converge, but the gap in aver- 
age wealth between whites and non- 
whites remained very large compared 
with differences in income, and the 
lower half of the wealth distribution for 
nonwhite households actually dropped 
behind the lower half for white house- 
holds — strongly suggesting that wealth 
inequality for nonwhites has grown 
faster than it has for the general popu- 
lation. 

Wolff says that age is clearly a pre- 
dictor of wealth inequality in the U.S. 

In 1983, the average marketable wealth 
of families under age 45 was far below 
the overall average — that of families 
over 45 was well above the average. 

And there have been important shifts 
in relative wealth across these groups. 
Middle-age households gained at the 
expense of younger and older house- 
holds. Racial differences are significant 
as well. Most of the gains in wealth for 
nonwhite households between 1983 
and 1989 were experienced by the up- 
per half of the distribution rather than 
those less affluent. Since 1970, for ex- 
ample, there has been no increase in 
the rate of home ownership among 
nonwhite families. While the gap in 
mean wealth between whites and non- 
whites closed further during the 1980s, 
the gap in median wealth widened, re- 
flecting a greater inequality in wealth 
for nonwhites. For the median non- 
white family, the differential is large 
and growing. 

An analysis of the composition of 
household wealth tells us about the 
ways in which households save. Be- 
tween 1962 and 1989, there was a ma- 
jor change, Wolff says, in the saving 
strategies of households. Savings from 
assets and equities declined from 52% 
of gross wealth to 38%, and savings 
from real estate and unincorporated 
business equity rose from 48% to 59%. 
Household debt as a proportion of net 
worth fell from 16.4% to 15% between 



1962 and 1983, increasing to 16.5% in 
1989. Owner-occupied housing was the 
most important asset in 1962, 1983 and 
1989, but its gross value was no more 
than one-third of total assets, or its net 
value more than one-fourth. In 1989, 
housing represented 29% of the gross 
value of assets, and net equity in own- 
er-occupied housing was only 20% of 
gross assets. Business equity and other 
real estate, and stocks, bonds and 
trusts exceeded gross house value as 
important assets. 

Other Societies 

Wolff’s analysis of data from Europe 
is unsettling in terms of our vision of 
American society as the primary model 
of social democracy. Although measures 
of household wealth and associated in- 
dicators of wealth inequality differ 
across nations, Wolff says the evidence 
suggests that in the early part of the 
20th century wealth inequality in the 
U.S. was comparable to that in Swe- 
den, but lower than in the United King- 
dom and the rest of Europe where the 
upper classes traditionally controlled 
most of the wealth. But by the late 
1980s this situation had been’reversed. 
The U.S. now has a much higher con- 
centration of wealth than the European 
Union. 

Wolff focuses his analysis on the 
United Kingdom and Sweden, because 
of the greater accuracy, comparability 
and availability of their data. What he 
finds is this: 

■ A dramatic decline in the degree of 
individual wealth inequality in the 
United Kingdom between 1923 and 
1974, with only minimal change 
from then on. The share of the top 
1 % of wealth holders fell from 59% 
to 20% in that period; by 1990 it 
was 18%. 

■ The same trend in Sweden , although 
the decline in wealth inequality was 
more continuous; by 1990 the share 
of the top percentile was 21%. 

■ A reduction in wealth concentration 
in the U.S., United Kingdom and 
Sweden until the late 1970s — but 
during the 1980s a sharp increase in 
the concentration of wealth in the 
U.S. and Sweden compared to the flat 
trend in the United Kingdom. This 
occurred in the context of conserva- 
tive economic policies in both the 



U.S. and United Kingdom, and a so- 
cialist government in Sweden. This 
leads Wolff to conclude that public 
policies in and of themselves likely 
do not explain similarities in the dis- 
tribution of wealth. 

Wolff made a separate study of 
France to compare the distribution of 
wealth between that country and the 
U.S., finding that the shares of the top 
1%, 5%, and 20% were significantly 
higher in the U.S., indicating that 
wealth inequality is greater. This ap- 
peared to be related to the fact that 
French households retain a much high- 
er proportion of their wealth in the 
form of owner-occupied housing, which 
is more equally distributed across the 
population than other assets. 

Income vs. Wealth Inequality 

Wolff wants us to understand that 
wealth inequality in the U.S. has al- 
ways been greater than income ine- 
quality. The top 1% of wealth holders 
has typically owned one-fourth of total 
household wealth, compared with the 
8% to 9% of share of income held by 
those at the low end of the income dis- 
tribution. Thirty-seven percent of the 
total real income gain between 1983 
and 1989 was realized by the top 1% 
of income recipients, compared to these 
recipients receiving 62% of the market- 
able wealth gain; 39% of the income 
gain went to the next 19% of income 
receivers, and 24% went to the bottom 
80%, compared with income receivers 
receiving only 1 % of the marketable 
wealth gain. Eventually the top fifth of 
the American population received more 
than 75% of the total increase in in- 
come, and all of the increase in wealth! 
The striking nature of these data reveal 
a worrisome chasm between social 
classes in American society. 

Historically, the distribution of 
wealth and the distribution of income 
have been in parallel. Income inequali- 
ty declined steadily during the Great 
Depression years and fell dramatically 
during World War II. A slight decline 
occurred between 1945 and 1953, fol- 
lowed by a sharp rise between 1981 
and 1989. In 1989 both the concentra- 
tion of wealth and income inequality 
peaked, the increase in both set a pre- 
cedent, with the exception of the 
1920s. Poorer households had seen 
their net worth decline at the same 
time their incomes were deteriorating. 
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Interestingly enough, where there 
have been discrepancies between 
wealth and income inequality, Wolff 
finds a cause in the ratio of stock prices 
to housing prices. Stocks accrue prima- 
rily to the upper class, housing to the 
middle class. When stock prices in- 
crease relative to housing prices, the 
share of wealth owned by the most af- 
fluent groups increases. Wolff contends 
that the data confirm this explanation. 

TheTaxation ofWealth 

In the U.S., household wealth is 
taxed by way of capital gains taxes and 
estate taxes. Individuals are exempted 
from taxation on the first $600,000 of 
wealth, and taxed at marginal rates be- 
yond that exemption. Capital gains, 
which reflect the difference between 
the selling price of an asset and its pur- 
chase price when an asset is actually 
sold, are taxed at 28% as part of the 
regular federal income tax system. But 
capital gains that are included in a de- 
ceased person’s estate are exempt. 

European countries have a much 
more extensive system for taxing house* 
hold wealth, incorporating a direct tax 
on wealth in addition to capital gains 
and estate taxes. The tax rates, howev- 
er, are not high — a few percents at 
most. In most countries, owner-occu- 
pied housing is taxable, as well as 
bonds, stocks and shares, and unincor- 
porated businesses. Most countries re- 
quire an annual assessment of the total 
market value of personal property, even 
though the definition of market value is 
usually difficult to determine. In addi- 
tion, most have a combination of 
death, gift, estate and capital gains tax- 
es. 

The Author’s Recommendations 

The remainder of Wolff’s book pro- 
vides data on the benefits of expanding 
wealth taxation in the U.S. Simulations 
suggest that a combined income-wealth 
taxation system may be more equitable 
than our current income tax system. 
Wolff claims that a wealth tax is sur- 
prisingly progressive, since the higher 
taxes would be born by older house- 
holds, married couples, and whites — 
that is, those most affluent. However, 
he pursues answers to several ques- 



O 



tions about the probable effects of 
wealth taxation, particularly revenue 
incidence and distributional effects. 
Regarding the first concern, he finds 
the Swiss version of a wealth tax the 
most effective in raising funds. With re- 
spect to the second, since income and 
wealth are not highly correlated, he 
sees a new wealth tax as shifting the 
tax burden away from young house- 
holds and toward more elderly ones, 
and from minorities to whites — again, 
those having benefited most and most 
able to contribute. His position is that 
wealth taxes have important equalizing 
effects on the distribution of income , 
and these effects increase with age , are 
greater for married couples than for sin- 
gles, and are stronger among white than 
nonwhite families. 

The author reiterates that “virtually 
all the growth in (marketable) wealth 
between 1983 and 1989 accrued to the 
top 20% of households,” while the bot- 
tom 40% of households experienced 
declining wealth. The disparity was 
made worse by the fact that a growing 
percentage of households had zero or 
negative net worth. This phenomenon, 
Wolff says, creates urgency about the 
need for new government policies. The 
1993 Congressional budget bill mirrored 
one strategy — an increase in marginal 
income tax rates for the wealthy. But 
these rates are still lower than at the 
beginning of the 1980s, and far lower 
than in the 1960s. And they are having 
only a modest effect on income ine- 
quality. 

Therefore the author pleads for a 
wealth taxation system similar to the 
Swiss direct taxation approach. He 
points to arguments in favor of such a 
system that go beyond its function as a 
revenue-producing and equalizing tool: 
1) income alone is not a sufficient 
gauge of well-being or the ability to pay 
taxes, and 2) an annual wealth tax 
may encourage individuals to transfer 
their assets from less productive to 
more productive uses. Wolff asks for a 
wealth tax fully integrated with a per- 
sonal income tax , as in Switzerland. Po- 
litically, he says, such a tax on wealth 
would affect only a very small propor- 
tion of the American population while 
making an important revenue contribu- 
tion. 
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Editor’s Note 



Readers interested in probing Edward 

Wolff’s writings for more insights, 
please see: 

Growth , Accumulation and Unproductive 
Activity: An Analysis of the Post-War 
U.S. Economy Cambridge University 
Press, 1987. 

As co-author, editor, or co-editor: 

Productivity and American Leadership: The 
Long View, MIT Press, 1989. 

The Information Economy: The Implications 
of Unbalanced Growth, The Institute for 
Research on Public Policy, 1989. 

International Comparisons of the Distribu- 
tion of Household Wealth, Oxford Uni- 
versity Press, 1987. 

International Perspectives on Profitability 
and Accumulation, Edward Elgar Pub- 
lishing Ltd., 1992. 

Convergence of Productivity: Cross- Na- 
tional Studies and Historical Evidence, 
Oxford University Press, 1994. 
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Balancing Growth 
with Economic 
Equality 

For another perspective on inequali- 
ty, we briefly review the main ideas 
and recommendations in Growth With 
Equity: Economic Policymaking for the 
Next Century by Martin N. Baily, Gary 
Burtless and Robert E. Litan, published 
by The Brookings Institution in 1993. 
The research supporting the book was 
sponsored by The Brookings Institu- 
tion’s Center on Economic Progress and 
Employment, which has published eight 
books and a series of Brookings Papers 
on Economic Activity since its estab- 
lishment in 1988. Growth With Equity 
is the final volume published by the 
Center. 

While the previous articles in this is- 
sue of Evaluation Forum concentrated 
our attention on economic and social 
inequality, these economists want us to 
look at the critical relationship between 
slow growth in the economy and in- 
creasing inequality. What concerns 
them is the long-term performance of the 
American economy. 

The authors report that in the twen- 
ty-five years following 1948, output per 
person grew faster than 2.5% a year. 
This was high enough to double the av- 
erage American’s living standard in less 
than thirty years. Since 1973, however, 
output has grown only 1.2% a year, 
which means that the average Ameri- 
can must wait twice as long to witness 
a doubling of his or her income. At the 
same time incomes have grown less 
equal. 

The American economic pie has 
been getting larger far more slowly than 
it did during most of the postwar era. 
And the pie has been sliced more un- 
evenly. In the next decade and be- 
yond, the nation faces no more urgent 
an economic challenge than reversing 
these two trends. 

A variety of problem-solving strate- 
gies have been suggested; Some ex- 
perts advocate for a Japanese-style in- 
dustrial policy emphasizing investment 
in specific industries and lines of pro- 
duction — through subsidies and trade 
protection . Others suggest new invest- 
ments in public infrastructure . Howev- 
er, the authors imply that their own 
analysis shows that these tactics may 



actually slow economic growth. Mean- 
while, they claim that Reaganomics 
contributed considerably to the in- 
crease in inequality. 

TheTrade-Off Between Efficiency 
and Equity 

A major theme of the book echoes 
Arthur Okun’s classic, Equality and Ef- 
ficiency: The Big TYadeoff published in 
1975 by Brookings. Baily, Burtless and 
Litan want us to understand that some 
policies that hurry long-run economic 
growth are not consistent with the goal 
of greater equity, and some policies 
that improve equity may not be consis- 
tent with successful economic competi- 
tion in the new global era. 

The authors use freer trade as an ex- 
ample of this dilemma. More open 
trade is likely to motivate companies to 
raise their productivity to remain com- 
petitive, but it may also reduce Ameri- 
can workers’ earnings and employment 
opportunities. Trade protection can re- 
duce the damage to workers, but may 
jeopardize economic growth. 

Despite the authors’ appreciation of 
Okun’s major tenet, they also believe 
that growth and equity can be 
achieved simultaneously. They refer to 
abundant evidence that progress to- 
ward both goals can be accomplished 
at the same time. In fact, they see poli- 
cies supporting both productivity and 
equity as being highly interrelated. In 
the 1960s, for instance, the U.S. experi- 
enced rapid growth and a sizeable re- 
duction in income inequality. Policies 
that increase the skill level of workers, 
reduce social problems, ameliorate con- 
ditions that lead to low earned income 
among the least skilled, all contribute, 
they say, to economic growth. And 
they are convinced that policies that 
free-up trade relations, and policies 
that protect and improve the job pros- 
pects of workers harmed by freer inter- 
national trade, can be pursued jointly. 

Balancing the Equation 

The authors acknowledge that pro- 
ductivity growth and improvements in 
living standards in an internationalized 
economy cannot occur without some 
level of worker displacement Macroeco- 
nomic policies can minimize unemploy- 
ment and stabilize inflation, but only if 
workers have the necessary skills to 
give them the flexibility required to 
move to new sectors of employment. 
Therefore they insist that public/private 
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policies must address the occupational 
training issue. 

Based on empirical evidence and the 
authors’ concerns, a comprehensive 
program is recommended — a mix of 
policies for encouraging long-run 
growth that does not sacrifice equity. 
Their agenda flows from their own re- 
search and from a series of studies on 
the slowdown in productivity in the 
U.S. and the generally poor perfor- 
mance of the U.S. economy. This agen- 
da responds to two major objectives: 

1. To increase economic growth as the 

main support for American living 

standards. 

What the authors mean is “a more 
rapid rate of advance in total 
output.. .and a faster rate of increase in 
output, holding constant the amount of 
inputs used to produce that output.’’ 
This can be achieved, they believe, by 
raising labor productivity. The economy 
could produce more by working longer 
hours and placing additional people in 
poorly-paid jobs, but the added income 
would be worth less than the free time 
given up. So it is wiser, the authors 
propose, for the labor force to “work 
smarter, produce more goods, and de- 
liver additional services, with the same 
level of work effort.” When the authors 
talk about economic growth, they 
therefore mean the growth in potential 
output, “the output that can be at- 
tained when labor and capital are fully 
employed and inflation is not accelerat- 
ing.” Consequently, a primary goal 
must be to improve the efficiency with 
which the nation’s labor and capital 
stock are utilized. This can be accom- 
plished, according to the authors’ agen- 
da, through advances in technology, im- 
provements in managerial skills, and the 
upskilling of workers . 

2. To reduce income inequality, par- 
ticularly wage earnings. 

The goal is to achieve greater equali- 
ty through a reallocation of the tax bur- 
den or through heavier reliance on gov- 
ernment transfers such as public 
assistance. The authors advocate deficit 
reduction, focusing both tax increases 
and spending cuts on the group that 
has benefited most from long-term eco- 
nomic growth, namely high-income 
Americans . The authors place more im- 
portance on increased equality in pre- 
tax incomes than on a redistribution of 
money via the tax code. In this sense, 
their strategy of choice is an increase in 
the job skills of low-wage workers . 
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Recommendations 

The authors’ comprehensive strategy 
involves a large number of recommen- 
dations under several policy areas. Un- 
derlying themes are: 

■ The most effective way to increase 
economic growth is to increase the 
investment portion of national out- 
put — that is, increasing private/ 
public spending that is oriented to 
future needs. 

■ The best strategy for equalizing in- 
comes is to increase individuals’ and 
families’ net savings, sufficiently 
that these savings rise faster than 
private/public investments. 



■ It is imperative that there be in- 
creased investment in worker train- 
ing, particularly for those lacking the 
kinds of labor market skills that are 
associated with higher-wage employ- 
ment. 

■ Free trade and its openness of for- 
eign goods, technology and capitol 
must be emphasized. 

■ Job losers must be afforded assis- 
tance in moving from shrinking to 
growing sectors of the economy to 
reduce the economic costs of dislo- 
cation. 

Some of the recommendations regarding 

labor policies are presented in Figure 1. 



Economic inequality: 
A Comparative View 

In 1995, the Organization for Eco- 
nomic Cooperation and Development 
(OECD) in Paris published a landmark 
report. Income Distribution in OECD 
Countries: Evidence from the Luxem- 
bourg Income Study. The report pro- 
vides dramatic information on postin- 
dustrial societies in which western 
youth are growing to maturity. The 
book-length report details the research 
questions, methodology and results as- 
sociated with a series of analyses of 
data collected by the Luxembourg In- 
come Study (LIS), supplemented by in- 
formation from studies carried out by 
member countries of the OECD. 

The LIS was begun in 1983 under 
the sponsorship of the government of 
Luxembourg and the Centre for Popula- 
tion, Poverty and Policy Studies in Wal- 
ferdange. It is being funded on a con- 
tinuing basis by the center and the 
International Network for Studies in 
Technology, Environment, Alternatives, 
and Development (INSTEAD), and by 
the National Science Foundations in 
member countries of the OECD. The 
main objective has been to create a da- 
tabase containing social and economic 
data gathered through household sur- 
veys in 25 countries, which can serve 
as a source of information for compara- 
tive studies. 

The OECD report is authored by 
three economists who are associates of 
the INSTEAD — Anthony B. Atkinson of 
Oxford University in the United King- 
dom, Lee Rainwater of Harvard Univer- 
sity, and Timothy M. Smeeding of Syra- 
cuse University — who directed the 
analyses. The studies were conducted 
in cooperation with OECD’s Economics 
and Statistics Department and its Edu- 
cation, Labour and Social Affairs Direc- 
torate. 

The purpose of the analyses was to 
examine income distribution across 
OECD countries and compare the ex- 
tent to which income is dispersed be- 
tween households, adjusting for the 
size of households, international differ- 
ences in real income, and (where suffi- 
cient data were available) the influence 
of property and employment incomes 
while accounting for direct taxes and 
cash transfers. 



Figure 1 ■ Edward N. Wolff’s Labor Policy Recommendations 



General Recommendations 

The improvement of skill training within 
schools. 



The improvement of the skills of the cur- 
rent labor force. 



The development of improved R&D pro- 
grams for public training efforts. 

The encouragement of firms to link work- 
er pay with company-level performance. 

The upgrading of employment services. 

The establishment of a non-cause-related 
program of earnings insurance. 



Specific Recommendations 
Encouragement to schools and employ- 
ers to provide high school transcripts to 
potential employers. 

The requirement that all students take na- 
tional standardized tests. 

The tying of salaries and promotions for 
teachers to the performance of students 
on standard tests. 

The establishment of a public/private enti- 
ty to develop tests and credentialing sys- 
tems for noncollege occupations. 

The institution of a pay-or-play training re- 
quirement for non-college-education 
workers. 

The development of a more systematic 
evaluation effort. 

The creation of a system for measuring 
performance. 

The infusion of employment services with 
updated occupational information. 

The conversion of the Ul system to a 
multi-use system for different kinds of un- 
employment. 
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The authors are quick to point out 
that not all the data available in the LIS 
databank cover the same time periods 
or are equally extensive. For eight major 
OECD countries, however, a substantial 
analysis of the level and trends in ine- 
quality was possible, and the results 
could be compared to national studies 
conducted within each of these coun- 
tries. The researchers also acknowledge 
that there were a number of expected 
barriers to data comparability. Never- 
theless, the LIS is viewed consistently 
by experts as the most comprehensive 
international study ever conducted in 
the industrialized nations. Furthermore, 
it avoids some of the definitional prob- 
lems in previous comparative analyses, 
and is highly relevant given the grow- 
ing interdependence of economic, so- 
cial and political trends in the countries 
involved in the study, and the compel- 
ling interest in cross-national research 
on income distribution. 



Trends Leading to Increased 
Policy Interest in Economic 
Inequality 

It is now critical to explore similari- 
ties and differences in industrial na- 
tions’ resource distribution patterns 
over time, and to understand how the 
market and demographic forces and 
public policies affect the relative eco- 
nomic status of various groups within 
these societies. 

Such issues directly influence the fu- 
ture opportunities and potential long- 
term life quality of youth in these coun- 
tries. Some of these influences had 
their origins outside government; others 
appear correlated with government pol- 
icies. The authors cite the following im- 
portant factors operating across OECD 
countries: 



■ The different experiences of these 
countries regarding macroeconomic 
performance, particularly in the 
1980s. 

■ The expanding inequality in the dis- 
tribution of earnings in several OECD 
countries. 



■ The general decline in the participa- 
tion of men in the labor force, and 
the increase in early retirement, at 
the same time female labor force 
participation has been increasing 
and female wages have risen relative 
to male wages. 

■ The aging of the population, creating 
strong budgetary pressures on na- 
tions’ safety nets. 
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■ The large-scale demographic chang- 
es occurring, such as the growth of 
single parenthood. 

■ The changing form of taxation and 
income transfer policy, such as the 
lowering of the top income tax 
brackets in several countries and the 
reduction in benefit coverage and 
levels. 

■ The world-wide increase in returns 
on capital income and changes in 
the distribution of asset ownership, 
including privatization programs. 

■ Immigration, technological change, 
and the transformation of Eastern 
Europe. 

Although the authors emphasize 
these changes, and independent stud- 
ies have suggested that all of these 
forces impinge on the distribution of in- 
come, influencing the nature and level 
of income inequality, they do not credit 
their analyses with estimating cause-ef- 
fect relationships. Accounting for the 
distributional effects of these trends, 
they say, is simply not currently possi- 
ble. 

The authors are straightforward also 
about barriers to data comparability: 

■ All comparisons were made in terms 
of intra-country relative measures, 
and did not have as a goal to seek 
absolute comparisons of levels of 
well-being in different countries. 

■ Certain microdata sets created prob- 
lems regarding “interior vs. exterior” 
comparability. 

■ Using information covering only two 
years in some cases made it difficult 
to identify general trends. 

In summarizing the section on data 
quality, the authors emphasize the fol- 
lowing: 1) full comparability is impossi- 
ble, 2) adopting a common set of defi- 
nitions means that estimates for any 
one country may be less satisfactory 
than if each were isolated, and 3) stan- 
dardizing through a common approach 
does not necessarily achieve compara- 
bility. Nevertheless, it is clear that the 
researchers cautiously identified the 
major problems and sought to adjust for 
them to the extent possible. 

The Main Findings from the 
Analyses 

Several approaches are used for re- 
porting comparisons across seventeen 
postindustrial nations. The general con- 



clusion is that the Scandinavian coun- 
tries, Benelux and Germany exhibited 
the least relative inequality. The highest 
levels of relative inequality were record- 
ed in the U.S., Switzerland and Ireland. 
There was a rise in the 1980s for the 
majority of countries studied, but this 
was not universal. Increases in mea- 
sured income inequality were largest in 
the Netherlands, Sweden, and particu- 
larly in the United Kingdom and the 
United States. 

The first comparative approach re- 
ported differences in median equivalent 
incomes per adult. For example, in the 
U.S. the median equivalent income per 
adult was $13,364 in 1986. For a family 
of four, this corresponded to a total in- 
come of $26,728. The equivalent in- 
come per adult at the lower quartile, 
25% up from the bottom of the income 
distribution, was $8,240; the 25th per- 
centile, expressed as a percentage of 
the median, was 61.7%. The decile ra- 
tio in the U.S. was the largest value re- 
corded in the income distribution for 
OECD countries. The bottom decile was 
only slightly over a third of the median, 
compared with around 45% in Australia 
and Canada, and values in excess of 
55% in Belgium, Finland, France, Ger- 
many, Luxembourg, Norway and Swe- 
den. However, differences across coun- 
tries were less dramatic at the top 
percentile. 

Another approach was to express 
differences in terms of what proportion 
of the population is below specified 
percentages of the median — for exam- 
ple, how many people are living in 
households with disposable incomes 
below half the median. If 50% of aver- 
age income is used as a standard defin- 
ing “low income,” and if “average” is 
measured by the median, then the U.S. 
stood out with 18.4% of the population 
living in households below 50% of the 
national median. The proportion with 
“modest” incomes, between 50% and 
70% of the median, was lowest in the 
U.S. and largest in the U.K. 

At the same time, in the U.S. the 
proportion in “well-to-do” households 
(above 150% of the median) was more 
than double that in Sweden. In the 
U.S., U.K., Italy, Ireland and Australia, 
about a quarter of the population is 
within the middle of the income distri- 
bution; in Sweden nearly 40% is in 
that range. In the U.S. there are more 
households in both the lower and up- 
per income categories; in Sweden, few- 
er in those categories. 
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A third approach was to report dif- 
ferences in terms of shares of total in- 
come. Using this measure, the U.S. 
again reveals considerable income ine- 
quality vis-a-vis other western societies. 
Average income in the U.S. would have 
to be higher by 50% if there were a 
more equal distribution of income . In 
looking at trends in the distribution of 
income over time, the majority of OECD 
countries showed a rise in inequality, 
most particularly the U.S. and the U.K. 

The Unique Methodology Used 

Cross-national comparisons are fre- 
quently misleading because the data do 
not reflect comparable definitions of 
the measures used to study key vari- 
ables. In the analyses of LIS and the 
supplementary national data, measure- 
ment issues were given unusual atten- 
tion. The major issues were 1) the defi- 
nition of income, 2) the definition of 
the unit and population studied, 3) the 
kind of adjustment made for household 



size , and 4) the kind of adjustment 
used to represent inequality. 

The researchers recognized that the 
strategies used were not a perfect solu- 
tion to the definitional problem. Never- 
theless, they were resolutions support- 
ed by the 1977 United Nation’s 
Provisional Guidelines. This directive 
outlined a standard structure for defin- 
ing and collecting income distribution 
statistics via national reporting systems. 
For example, the authors acknowledged 
that the line between market and gross 
income was sometimes artificial. The 
only serious concern remaining was the 
exclusion of employers’ contributions 
from wages and salaries to government 
occupational transfer programs, and 
rental income from owner-occupied 
housing. Also, measures of some vari- 
ables were not available or complete. 
For instance, infrastructure, defense 
and protective services were excluded, 
as well as health, education and other 
in-kind subsidies. And the analyses 



counted only direct taxes such as na- 
tional income taxes and payroll taxes 
paid by employers and the self-em- 
ployed. 

Some Conclusions 

Despite understandable data quality 
and comparability problems, the OECD 
analyses take a giant step toward a bet- 
ter understanding of the differences in 
income inequality across postindustrial 
societies. The comparisons make a 
statement about the relationship be- 
tween social democratic principles and 
economic equity. This statement is 
alarming with respect to the inequali- 
ties evidenced in the U.S. in the late 
1980s, given the social polemics about 
egalitarianism. It is also an important 
commentary on our form of capitalism 
and the role of the central government. 
And it provides a context for discussing 
problems facing American youth, which 
should be taken very seriously. 



From a 1990s report prepared by the Office ofTechnology Assessment, U.S. Congress 



In 1993, approximately 7.3% of the 
U.S. population was young adoles- 
cents, ages ten through fourteen. Of 
these, approximately 20% were liv- 
ing below the federal poverty line. 
By the year 2000, more than one- 
third of all young adolescents will 
be members of racial or ethnic mi- 
norities. To compete in the global 
economy of the twenty-first century, 
America will need all of its young 
people to be healthy and well edu- 
cated. 

The average proficiency in sci- 
ence, mathematics, and writing 



among eighth graders was slightly 
higher in 1992 than it was in the 
1970s. However, these performanc- 
es have not improved enough to 
keep pace with the higher level of 
skills required in a global economy. 
In 1992, only 28% of eighth graders 
attained the proficient level in read- 
ing, representing solid academic 
performance at grade level. Two per- 
cent read at or above the advanced 
level, representing superior perfor- 
mance. The remaining 70% demon- 
strated partial mastery of the knowl- 
edge and skills required for pro- 



ficient work in the eighth grade. In 
1990, 7% of the eighth grade class 
of 1988 (most of whom were then fif- 
teen and sixteen) were dropouts. By 
their senior year, 12% of this class 
were dropouts. Dropout rates vary 
by students’ race/ethnicity: white 
(9.4%); black (14.5%); Hispanic 
(18.3%); Asian/Pacific Islanders 
(7%); and American Indian (15.4%). 
The personal economic conse- 
quences are profound. High school 
dropouts are unlikely to earn suffi- 
cient wages to support themselves 
or their families. 
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Introduction 



In this issue we place a magnifying 
glass over youth issues. We have 
asked the following group of experts 
on youth development, programming 
and evaluation to respond to a set of 
questions about youth problems and 
possibilities. 

David E. Brown is a senior policy 
analyst with the Center for Policy Re- 
search at the National Governors’ As- 
sociation. Long interested in youth is- 
sues, beginning with a high school 
internship with one of New York’s Youth 
Bureaus, he majored in Political Sci- 
ence and Urban Affairs at American 
University and received a graduate de- 
gree in public administration from 
Baruch College as a National Urban 
Fellow. 

In the early 1980s, Brown adminis- 
tered federally-funded youth employ- 
ment programs in New York and later 
coordinated education and work expe- 
rience projects targeting economically 
and educationally disadvantaged out- 
of-school youth in New York City. In the 
late 1980s, he developed an employer- 
based career exploration and job 
preparedness program for at-risk teen- 
agers. Working for the Maryland De- 
partment of Juvenile Services , he for- 
mulated policies and legislation to 
improve the responsiveness of the 
state’s juvenile justice system. At the 
Center for Policy Research, Brown is 
concentrating on social policy, legisla- 
tion and new initiatives in youth develop- 
ment, school-to-work, job training, youth 
violence, and community service. 

Gary Burtless is a Senior Fellow in 
the Economic Studies program at the 
Brookings institution in Washington, 
D.C. A Yale and MIT-trained economist, 
Burtless held positions in both the U.S. 
Department of Labor and the U.S. De- 
partment of Health, Education and Wel- 
fare in the late 1970s and early 1980s 
— in the office of the Assistant Secre- 
tary for Policy, Evaluation and Re- 
search in DOL, and the Office of the 
Assistant Secretary for Planning and 
Evaluation in HEW. He has been at 
Brookings since 1981, with the excep- 
tion of 1993 when he served as Visiting 
Professor of Public Affairs at the Uni- 
versity of Maryland. 

O 
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His major areas of concentration 
have included labor market issues, in- 
come distribution, and public tax and 
transfer policies. His recent research 
has focused on the sources of increas- 
ing wage and income inequality in the 
U.S., the influence of international 
trade on income inequality, the job 
market prospects of public assistance 
recipients, and the reform of social in- 
surance in Central and Eastern Europe 
and developing countries. During his 
distinguished tenure at Brookings, he 
has co-authored and edited several 
books and authored numerous articles 
on critical social policy issues. 

Susan P. Curnan has been director 
since 1990 of the Center for Human 
Resources in the Heller Graduate 
School for Advanced Studies in Social 
Policy at Brandeis University. She is 
also an associate professor and chair 
of the Heller School’s graduate pro- 
gram in Management of Children, 

Youth and Family Services. Her major 
work is in translating research and ex- 
pert opinion into “best practices,” the 
design and management of innovative 
national and local evaluations, and 
consultation on issues such as educa- 
tional reform, workforce preparation 
and community development. Her inter- 
ests extend also to strategic planning 
regarding youth issues and the moni- 
toring of a clearinghouse and national 
electronic network on children, youth 
and families. 

Curnan has been principal investi- 
gator of a four-year youth research 
project funded by the U.S. Department 
of Labor which synthesized research 
findings from a number of social sci- 
ence disciplines. She was principal 
consultant in the conceptualization and 
implementation of the Department of 
Labor’s Youth Fair Chance initiative, 
and was chief developer of the Sum- 
mer Beginnings National Work and 
Learning Network. In 1982 she was 
elected an Outstanding Young Woman 
in America in recognition of her profes- 
sional and community service accom- 
plishments. 

Andrew B. Hahn is Human Services 
Research Professor and Associate 
Dean of the Heller Graduate School for 
Advanced Studies in Social Welfare at 
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Brandeis University. He is also the se- 
nior faculty member associated with 
the Center for Human Resources, the 
director of the Heller Program on Inno- 
vations in Social Policy, and the direc- 
tor of the Rothman and Giddon Fami- 
lies Project on Media, Health and 
Social Policy. Most of his research and 
policy work has been in the areas of 
education, training and employment for 
disadvantaged youth, dropout preven- 
tion, and welfare reform. He was co-di- 
rector of Career Beginnings, a national 
mentoring program for disadvantaged 
youth, and is involved currently with an- 
other large-scale youth project, Summer 
Beginnings. 

Hahn has evaluated a range of 
youth initiatives for the Rockefeller, 
Ford, Clark and Lilly Foundations, re- 
cently completing an evaluation of the 
Quantum Opportunities program which 
focuses on youth development. He has 
been principal investigator for a num- 
ber of multi-year national youth pro- 
grams, and is currently principal evalu- 
ator for a national business/education 
partnership program, a national net- 
work of education and entrepreneur- 
ship programs, a high school program 
in Baltimore, and a project to examine 
federal youth policies. 

Robert J. Ivry is Senior Vice Presi- 
dent for External Affairs and Develop- 
ment at the Manpower Demonstration 
Research Corporation (MDRC). He is 
an expert on a range of policy issues, 
particularly youth employment policy. 

At MDRC he is responsible for devel- 
oping and implementing many of the 
research firm's major demonstration 
projects, and for disseminating the re- 
sults and recommendations flowing 
from evaluations of these demonstra- 
tions to policymakers and practitioners. 
He has been involved with a number of 
youth demonstrations such as JOB 
START, New Chance, Career Begin- 
nings and Project Transition — their 
development, implementation and 
evaluation. He was instrumental in 
broadening MDRC’s research focus to 
include the evaluation of innovative 
school-based reforms beginning with 
the School-to-Work Transition Study. 

And he led the team conducting an ex- 
perimental evaluation of Career Acad- 
emies, the first such study of one of the 
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options under the school-to-work legis- 
lation. 

Ivry’s career has combined project 
management with policy research. In 
the 1970s he managed the mayor of 
Baltimore’s Youth Services. Moving to 
MDRC in 1980, he took part in the 
management of the 1980s’ Youth Incen- 
tive Pilot Projects, the largest national 
youth employment demonstration ever 
funded by the U.S. Department of La- 
bor. He has since been involved with a 
range of policy initiatives in education 
and employment for disadvantaged 
youth, school dropouts, teen parents 
and youth in welfare families. 

Christopher T. King is Research 
Scientist and Associate Director of the 
Center for the Study of Human Re- 
sources in the Lyndon B. Johnson 
School of Public Affairs, the University 
of Texas at Austin. At the Center since 
1980, King has conducted numerous 
research projects in the areas of edu- 
cation, training, welfare and employ- 
ment. His current research focuses on 
the implementation and impacts of the 
JOBS program and related welfare ini- 
tiatives, and on outcome measurement 
and school-to-work issues. 

He has taught labor economics, hu- 
man resource management and policy 
analysis at the University of Texas, 
Southwest Texas State University and 
the University of Utah. He was Chief of 
Research, Demonstration and Evalua- 
tion for the Texas Department of Com- 
munity Affairs and the Governor’s 
Office, initiating an experimental evalu- 
ation of Texas’ dislocated workers pro- 
gram. In addition he was principal edi- 
tor for the U.S. Department of Labor’s 
Advisory Committee report, Working 
Capital , and authored Cross-Cutting 
Performance Management Issues in 
Human Resource Programs for the Na- 
tional Commission for Employment Pol- 
icy. He has served as a consultant on 
performance measurement and man- 
agement issues for a number of gov- 
ernment and private sector agencies 
and organizations. 

Gary Walker served as president of 
Public/Private Ventures in 1995, and as 
its executive Vice President between 
1986 and 1994. Public/Private Ven- 
tures is a private nonprofit research 
and demonstration organization in 
Philadelphia specializing in youth pro- 
gramming and evaluation. He was for- 
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merly founder and partner of Grinker, 
Walker and Associates, Inc. in New 
York, a private sector research firm. Be- 
tween 1981 and 1983 he was Senior 
Vice President for program develop- 
ment at the Manpower Demonstration 
Research Corporation in New York, 
and held other managerial positions in 
MDRC between 1974 and 1983. 

A graduate of Yale Law School, 
Walker began his career with a New 
York law firm, then became president of 
a legal service firm, and subsequently 
served in supervisory positions in the 
Vera Institute of Justice in New York. He 
is the author of a number of policy 
analyses and evaluations focusing on 
youth, and a consultant to The Ford 
Foundation and other national founda- 
tions. He currently serves on The As- 
pen Institute Roundtable on Compre- 
hensive Initiatives for Children and 
Families, and is a member of the Urban 
League’s Advisory Committee on Youth 
Development. 

Alan Zuckerman is Executive Direc- 
tor of the National Youth Employment 
Coalition, which involves eighty mem- 
ber organizations. The Coalition’s pur- 
poses are to improve training capacity 
and prepare youth to become produc- 
tive workers, to serve as a clearing- 
house for information about successful 
youth development programs, to pro- 
vide a forum for exploring alternative 
youth employment policies and strate- 
gies, and to create a support network 
for the youth development organiza- 
tions. 

Zuckerman has developed policies 
and implemented programs in employ- 
ment and training for the U.S. Depart- 
ment of Labor, the Council of the 
Southern Mountains, the Opportunities 
Industrialization Centers of America, 
and the Home Builders Institute — and 
Temple University, where he has 
taught courses in manpower and youth 
development. He has been a consult- 
ant to the governors of New Jersey and 
Pennsylvania, the President’s Commit- 
tee on Manpower, the National Com- 
mission for Employment Policy, com- 
munity development agencies and 
public housing authorities. A long-time 
advocate for improved efforts to assist 
young people in developing their 
potential, Zuckerman has written nu- 
merous articles on youth employment, 
service delivery and computer commu- 
nication. 



Expert Panel 
Responses 

Question: What major problems do 
you feel American society is facing in 
the 1 990s regarding youth development , 
education , occupational training and 
employment? 

David Brown 

Look at this quote: 

What do our children and youth 
face today? Frustration, insecuri- 
ty, fear, despair, resentment, con- 
fusion, and lack of opportunity. 

We must not let these difficult 
years rob our young people of 
hope. 

This statement by the General Secre- 
tary of the National Child Labor Com- 
mittee describes contemporary circum- 
stances, yet it was made on April 26, 
1939 during the Conference on Children 
in a Democracy. This national meet- 
ing — hosted by the first U.S. Secretary 
of Labor, Frances Perkins — was 
opened by President Franklin D. 
Roosevelt and First Lady Eleanor 
Roosevelt. The ominous rise of fascism 
and communism across the globe in the 
1930s and the combination of wide- 
spread youth idleness, unemployment 
and despair in America heightened 
anxieties about the commitment of 
American youth to democratic princi- 
ples. However, many of the pressing 
youth problems addressed by the 1939 
conference were ameliorated in large 
part by the onset of World War II. Most 
young people were, for better or worse, 
actively engaged in the war effort. Sub- 
sequently, many of the young of that 
era who survived the war were the 
beneficiaries of the greatest single fed- 
eral investment in postsecondary edu- 
cation — the GI Bill — and the unprec- 
edented postwar economic expansion. 

Today, many of our young people are 
once again facing frustration, insecuri- 
ty, fear, despair and resentment. With- 
out promising prospects for their fu- 
tures, hope has again become an 
increasingly rare commodity among 
many young adults, particularly those 
residing in urban and rural communi- 
ties. Increasing levels of youth crime 
and violence, and the seemingly inex- 
plicable allure of gangs and paramilitary 
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hate groups among many youth, are 
indicative of a prevalent loss of hope. 
States have experienced a continued 
rise in the social and fiscal costs of the 
nation’s inability to effectively absorb 
many of its youth into the economic 
and cultural mainstream. Drug-related 
murders, gang activity and malicious 
hate crime have plagued communities. 
High rates of teenage pregnancy con- 
tinue to arrest the development of 
many adolescent girls, foster irresponsi- 
bility among teen fathers, and impair 
the development of the next generation 
of children. 

This time, however, there is no na- 
tional challenge as formidable as World 
War 11 or a federal program as ambi- 
tious as the GI Bill on the horizon capa- 
ble of once again positively engaging 
vast numbers of the nation’s young 
people. Furthermore, America is no 
longer the singular economic power- 
house it was after the war. Today, U.S. 
companies must compete aggressively 
with their foreign counterparts to retain 
their share of international trade. 

Gary Burtless 

Young people in the United States face 
one main problem in getting estab- 
lished in the work force — finding a 
source of training that is widely recog- 
nized as effective. An increasing per- 
centage of young people attempt to 
solve this problem by enrolling in high- 
ly-regarded two and four-year colleges. 
This solution probably works for the 
majority of students who enroll and 
complete college. But for many teenag- 
ers college is not a satisfactory solution 
to their problem. They may have failed 
to perform well in secondary school 
classrooms, so it is doubtful whether a 
college classroom will offer a better 
venue for training. What these young- 
sters need is a source of hands-on oc- 
cupational training that is highly re- 
garded by employers. There are very 
few sources of such training outside 
the college classroom. 

Youngsters who do not fare well in 
college do not have good training op- 
portunities in the private workplace. 
Few employers are willing to take the 
risk of investing in high school dropouts 
or high school graduates who are in 
their late teens or early twenties. As a 
result, many job market entrants who 
lack a college degree are destined to go 
from job to job without obtaining much 
occupational training in any of them. 



There can be little doubt that this 
school-to-work "system” leaves many 
youngsters unprepared to earn a good 
living. 

Andrew Hahn 

The major problem we face regarding 
the “human resource” challenge is that 
as a nation we have not and do not 
take seriously the development of a lo- 
cal infrastructure to help young people 
make developmental^ appropriate tran- 
sitions into the mainstream of society. 
The job training/youth development/ 
second chance education "field” is real- 
ly a constellation of small professional 
efforts operating on the margin of most 
communities’ efforts. While this gener- 
alization certainly has many exceptions, 
the fact is that we do not take skills 
training in the public and not-for-profit 
spheres very seriously. It is an under- 
capitalized field serving only a fraction 
of the universe of need and eligibles. 
There are few professional certification 
standards, and program quality mea- 
sures are rarely widely used and ac- 
cepted. Evaluations of programs, with 
some important exceptions, continue to 
reinforce the image that these programs 
are ineffective and do not matter. 

Skill training for the disadvantaged 
has always been a small enterprise. It 
is now in retreat, and even among 
some advocates there is the feeling that 
it might be better to experiment with 
consumer choice models (e.g., skill 
grants) than invest in "programs.” Pub- 
lic education enjoys a property tax form 
of financing; in job training the finan- 
cial base and very mission of the enter- 
prise is still debated. This shows how 
much work is still required to get work- 
force development policy and programs 
firmly embedded in our local communi- 
ties. 

Rob Ivry 

As we approach the next millenni- 
um, the state of America’s youth is pre- 
carious. Stories abound in the press 
about youth alienation and apathy, ju- 
venile crime, stagnant or deteriorating 
test scores, persistently high dropout 
rates, youth joblessness, and teenage 
pregnancy. The largely discouraging re- 
sults from studies of youth employment 
and training programs have overshad- 
owed the success stories and are often 
generalized to indict all social invest- 
ments for young people. Proposed poli- 



cies to block grant federal welfare and 
workforce development dollars to states 
and localities is taking the country into 
uncharted waters and runs the risk of 
exacerbating these problems. The unin- 
tended consequences for the health 
and economic well-being of young peo- 
ple could be devastating. Youth pro- 
grams are likely to get caught in the tri- 
ple squeeze of reduced funding, 
non-targeted funding for youth from 
low income families, and the competi- 
tion for resources between programs for 
adults and programs for youth. Since 
young people are not a voting constitu- 
ency, they are likely to get short- 
changed on resources and services. 

One of the challenges youth advo- 
cates face is to change the public 
mindset, to view young people as a re- 
source and asset to be nurtured and 
cultivated rather than a liability to be 
ignored or vilified. What often gets ob- 
scured in the blizzard of negative youth 
statistics is the incredible resiliency of 
young people, despite overwhelming 
circumstances. For example, contrary to 
the stereotype of teen parents, there is 
evidence that many young mothers on 
welfare remain in school, work and en- 
roll in training programs despite facing 
formidable personal and situational 
problems. Public support for education 
and crime prevention provide a rallying 
point for policymakers to overcome 
their shortsightedness and enact poli- 
cies which support the development of 
young people from adolescence to 
adulthood and from school to work. 

There are several compelling reasons 
why investment strategies are impera- 
tive. First , there is a broad-based con- 
sensus that our nation’s productivity 
and international competitiveness hinge 
on helping young people — our future 
labor force — become skilled workers. 
This is one of the reasons why the 
School-To-Work Opportunities Act of 
1994 passed with bipartisan support, 
and why the nation’s governors agreed 
at the education summit meeting in 
March 1996 to encourage states to set 
skill standards. Second, the public costs 
associated with disconnected youth, in 
terms of outlays, lost productivity and 
reduced taxes are exorbitant. Third, 
the public investment in knowledge de- 
velopment over the past twenty years 
has the potential to pay dividends if 
the terms of the debate can be 
changed from “why have programs 
failed?” to "how can youth programs 
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be more successful?” While the results 
of most youth programs have been dis- 
couraging, especially for young people 
who have dropped out of school, there 
are some beacons of proven effective- 
ness. The challenge is to promote poli- 
cies which enable states and localities 
to build upon and expand these suc- 
cesses while encouraging further exper- 
imentation, so that the repertoire of 
proven programs can grow. 

Chris King 

Undoubtedly, every generation of 
youth has had to face its own unique 
set of major problems since the begin- 
ning of time. In this century, they have 
had to deal with massive urban migra- 
tions and the disappearance of rural 
family life, an all-encompassing eco- 
nomic depression, world wars and divi- 
sive “conflicts.” This generation is now 
encountering its particular set of prob- 
lems. Coming as they do after a 
uniquely prosperous era in our history, 
these problems must appear especially 
daunting. 

First among these problems is the 
fact that a job no longer carries with it 
what it did for these young people’s 
parents and their parents’ parents. For 
most, a job once meant a career, a 
sense of identity and belonging, a 
steady rise in earnings — and an array 
of associated health, vacation and other 
benefits, as well as long-term economic 
security at work’s end. Getting a good 
education, working hard and playing 
by the rules brought these things and 
more. Today, young people are watch- 
ing as most of this picture fades from 
the screen within a time so short they 
can catch it even with the shortest of 
attention spans. 

Peter Capelli (“Rethinking Employ- 
ment,” British Journal of Industrial Re- 
lations, forthcoming) and others have 
suggested that career paths by occupa- 
tion have become truncated, and 
where they still exist their beginning 
rungs are no longer inside the firm. 
Rather they are located within tempo- 
rary services. Many young (and not so 
young) workers now find that they 
must set out on their careers working 
as contingent workers hoping to secure 
a more steady second-rung job with a 
“real” employer. Moreover, as they go 
on, employers are no longer providing 
the training which once accompanied 



these jobs, nor are they maintaining the 
institutional memory of them which 
older workers brought to bear, so that 
young workers have access to the type 
and level of human capital investment 
tools within firms which they would 
have had before. 

Second, and related, is that even 
most workers who have been holding 
onto their jobs are watching as their 
earnings decline in value. A now well 
documented and expanding gap exists 
between those in the top reaches of 
the labor market and virtually everyone 
else (Burtless, Danziger, Gottschalk, 
and many others). Youth must be fac- 
ing increasing pressure in poor families 
to drop out of school to help make ends 
meet, especially in light of the dimin- 
ishing and less dependable safety net. 

Third, whether at the national, state 
or local level, government budgets are 
severely squeezed. Not surprisingly, 
governments are attempting to divest 
themselves of, or cut back on, some of 
their primary functions. The enormous 
and unprecedented peacetime deficits 
of the 1980s are at the heart of this 
phenomenon. The recent school-to- 
work initiatives are a prime example of 
this trend. The federal government es- 
sentially has promised only “seed” 
money for this initiative, explicitly ex- 
pecting the financially-strapped states 
and localities to step in with full subse- 
quent funding. This is despite wide- 
spread recognition that poor and un- 
structured transitions from public 
school to higher education or to work 
on the part of our youth is truly a na- 
tional problem of considerable magni- 
tude (Commission on the Skills of the 
American Workforce’s 1990 report, 
America’s Choice; and Ray Marshall 
and Bob Glover, 1994). 

Fourth, the nation seems to be mov- 
ing into a period of intolerance and 
mean-spiritedness. This is unprecedent- 
ed in times of high growth in Gross Do- 
mestic Product (GDP) and productivity, 
and relatively low levels of unemploy- 
ment and inflation. Any number of 
groups appear to be singling out minori- 
ties, homosexuals, immigrants, welfare 
recipients and single mothers and their 
families as objects of scorn, derision 
and out-and-out hatred. Ironically, 
national and state executives and legis- 
latures are rapidly rushing to hold indi- 
viduals and families “personally respon- 
sible” for their social and economic 
misfortunes, even as the world’s econo- 



mies have so clearly become inter- 
meshed at all levels and in all dimen- 
sions, and people’s control over their 
lives diminished. Note that some of the 
fastest growing groups in our country, 
now and for the foreseeable future, are 
the very ones who are the focus of this 
intolerance. 

Educational institutions are now ex- 
pected to play roles which would have 
been inconceivable just a generation or 
two ago. Schools must educate for de- 
mocracy and citizenship as well as 
playing a major role in preparing youth 
for work and careers. They also are 
serving as health and parenting skills 
educators, child care facilities and so- 
cial halls. 

Gary Walker 

For a large and increasing number of 
youth, the basic forces in their “natu- 
ral” surroundings — family, neighbor- 
hood, friends, school, employers, con- 
nections to the larger world — are 
weakening simultaneously. Quite unwit- 
tingly, our society has become steadily 
less hospitable to the healthy develop- 
ment and transition to adulthood of its 
youth. That is the most fundamental 
problem we face. 

In addition, our society has in its 
historical and cultural roots a deep am- 
bivalence about direct government in- 
tervention in social issues. That ambiv- 
alence is particularly evident regarding 
social and economic equity policies. We 
do not consistently exercise the com- 
mitment or good sense necessary for 
government policies and resources to 
help rebuild the basic positive forces in 
many youths’ lives. We have a hard 
time seeing public social policy as an 
integral part of what it takes to keep a 
complex free market and individual-ori- 
ented society hospitable to healthy 
youth development. Worsening condi- 
tions for youth, and an ambivalent atti- 
tude about using public policies and 
funds to improve those conditions, are 
a serious pair of obstacles. 

Alan Zuckerman 

Andrew Sum, of Northeastern Uni- 
versity, observes that economic adoles- 
cence has been extended until the age 
of thirty. This is happening at a time 
when wages for all but the top 20% of 
wage earners are going down and 
young people are working fewer hours. 
As a result, conventional wisdom about 
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young people’s entrance into the labor 
force must be tempered by the cruel re- 
ality of the contemporary labor market. 
According to Dr. Sum: 

■ There are 3 million dropouts ages 16- 
24 — 2 million dropouts are unem- 
ployed. 

■ The average number of weeks 
worked by 17-21 year olds has 
dropped slightly for all young people 
and plummeted for African Ameri- 
can male dropouts to 20.9 weeks a 
year. (The average for African Ameri- 
can male high school graduates is 
35.3 weeks compared to 40.5 weeks 
for all high school graduates.) 

■ Real annual earnings for 17-21 year 
olds fell between 1973 and 1994 by 
38% (to $4,968) for male dropouts 
and 25% for all male high school 
graduates (to $10,180). 

■ For 20-29 year olds, mean earnings 
for male dropouts fell 42% (from just 
under $20,000 in 1973 to $11,223 in 
1994). 

■ According to Doug Besharov of the 
American Enterprise Institute, 10% 
of America’s youth (ages 16-23) are 
“disconnected” — not in school, not 
in the military, not working, not mar- 
ried for at least one year. 

■ For women, the rate is 34% of Afri- 
can Americans, 21% of Hispanics, 

9% of whites. 

■ For men, the rate is 23% of African 
Americans, 13% of Hispanics, 7% of 
whites. 

■ The long term impact of being a 
“disconnected youth” is: 

- 50% are poor. 

- Income 50% lower than never- 
disconnected. 

- Less likely to have health insur- 
ance. 

- Less likely to have ever worked 
for a full year. 

Lower wages and a later entry into 
full-time employment makes it difficult 
to achieve the American Dream — get- 
ting a job, raising a family and living 
happily ever after. For too many young 
people, work is sporadic and low pay- 
ing, leading nowhere. This reality de- 



stroys motivation and limits aspirations. 
Without faith in the future, many 
young people drop out of school and 
fail to invest in building their work-re- 
lated skills. 



Question: What socio-economic, polit- 
ical, demographic and cultural trends 
in American society do you think have 
contributed most to the development or 
exacerbation of these youth problems? 

David Brown 

Over the past few decades, the na- 
tion has responded to the problems of 
its most vulnerable youth by initiating 
myriad crisis-driven and narrowly-fo- 
cused prevention and intervention pro- 
grams. A recent General Accounting Of- 
fice report asserts that “The federal 
response to the problems of delinquent 
and at risk youth is currently manifest- 
ed in 131 separate programs adminis- 
tered by 16 departments and agencies. 
Many of the programs are authorized to 
fund multiple services and are targeted 
towards multiple youth groups, creating 
the potential for program overlap and 
duplication of services.” 

Most of these programs target de- 
fined segments of the nation’s young 
people: those exhibiting behaviors or 
experiencing problems commonly asso- 
ciated with a specific adolescent “pa- 
thology” (e.g., delinquency, substance 
abuse and pregnancy), or those 
deemed at risk for developing such be- 
haviors or problems. Interventions fre- 
quently occur after a problem has sur- 
faced, rather than seeking to prevent 
the problem from occurring in the first 
place. They often fail to address the 
psychosocial developmental needs of 
youth. This categorical, uncoordinated 
and fragmented maze of federal pro- 
grams confounds local service delivery 
and promotes delayed intervention rath- 
er than holistic prevention. Moreover, 
these programs have not significantly 
reduced the problem associated with 
adolescents. In many communities, the 
problems have increased despite inten- 
sive interventions. 

Since the dawn of this century, we 
have continued to postpone the age at 
which young people are expected to as- 
sume adult roles. Yet one of the most 
commonly overlooked developmental 



imperatives of adolescence is the need 
to be valued, to have a useful, mean- 
ingful and respected role. Since in 
many instances the family and the 
community do not provide such oppor- 
tunities, young people seek and find 
validation, status and respect from their 
peer group. However, when unguided 
by parents or other adults, the norms, 
values, expectations and behaviors of 
the peer group often undermine years 
of parental socialization. For most 
youth, it is within the peer group that 
relationships with the opposite and 
same sex are developed, new roles are 
tested, status is achieved, indepen- 
dence asserted, and identity eventually 
established. Concurrently, much of the 
antisocial and seif-destructive behavior 
engaged in by adolescents occurs with- 
in the context of peer group activities 
or before a peer audience. These be- 
haviors, though often considered devi- 
ant by adult standards, generally fall 
within the accepted boundaries of the 
peer group’s norms. The need to be ac- 
cepted and to achieve status among 
peers has proven to be a powerful and, 
to date, virtually unchallenged force 
during adolescence. 

The continued weakening of family 
and community social support systems, 
coupled with the dearth of programs or 
strategies promoting overall youth de- 
velopment, have placed many of Ameri- 
ca’s young people “at risk.” Due partly 
to family breakdown and increased 
feelings of alienation, many youth are 
engaged in self-destructive behaviors, 
including delinquency, substance 
abuse and early childbearing. Further- 
more, increasing numbers of the na- 
tion’s youth are growing up in poverty 
and with dysfunctional families or sin- 
gle-parent families. Many have little 
hope for a better life and limited oppor- 
tunities to develop to their full poten- 
tial. As the prospects for decent-paying 
jobs have eroded for many poorly edu- 
cated youth, so has their faith in the 
system. However, the increasing reli- 
ance of states on secure confinement to 
address the failures of our youth-serv- 
ing systems, coupled with the cuts in 
funding for preventive programs, are 
depleting state resources, just as the 
nation’s youth population is about to 
surge again in the latter part of the de- 
cade. 
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Gary Burlless 

The problems faced by non-college- 
bound youngsters have been exacerbat- 
ed by the strong growth in demand for 
highly qualified workers. This trend is 
evident in widening wage disparities 
between workers with college degrees 
and workers who have not received any 
education beyond high school. The 
trend is also evident in widening dis- 
parities between the wages received by 
workers who have just entered the la- 
bor force and workers who have had 
fifteen or more years of job experience. 
Both educational qualifications and 
lengthy work experience are more high- 
ly prized today than they were two or 
more decades ago. 

It is more important than ever that 
workers obtain good occupational cre- 
dentials, but young workers without a 
college degree find it harder than ever 
to find employers willing to invest in 
the kinds of credentials that actually 
would help their young workers. Young 
people with less than a college degree 
have paid a heavy price for this defi- 
ciency in our training system. They 
have suffered steep wage declines in 
comparison with youngsters who had 
similar levels of education but who en- 
tered the work force twenty or more 
years ago. This has put heavy pressure 
on parts of the American social safety 
net that serve young adults and their 
children. Over time, the nation has 
seen growing caseloads for food 
stamps, AFDC, Medicaid, and other 
public benefits targeted on poor fami- 
lies with children. 

Andrew Hahn 

I am not particularly skillful about 
identifying the trends that have con- 
tributed the most to the development 
of social problems and the exacerbation 
of youth problems. Professor William 
Julius Wilson has discussed the outmi- 
gration of middle class people from in- 
ner cities; the rise of a highly estranged 
group of people isolated in poor neigh- 
borhoods; the changing structure and 
location of jobs — not to mention the 
unintended effects of affirmative action 
and civil rights, which may have con- 
tributed to middle class flight, and the 
changing structure of families with the 
associated challenges of single parent- 
hood and illegitimacy. I think life is too 



complex to disaggregate genuinely 
these relationships, although social sci- 
entists enjoy trying to do so. I do think 
a trend worth 'noting is how the pre- 
ceding social and economic movements 
have touched the program sphere. Con- 
sider that too many talented people 
have outmigrated to the suburbs, afraid 
to work with young people in our inner 
city neighborhoods. Program staff in 
workforce and community programs are 
often disconnected from highly es- 
tranged youth, and do not know how 
best to help them combine hard skill 
training with soft and life skill training. 
The "background" problems of family 
life and poverty are so severe that 
many young clients bring a bundle of 
problems that "single purpose” short- 
term programs have a difficult time ad- 
dressing, The point is that the same 
trends that can be identified for the 
country as a whole affect workforce de- 
velopment programs. The latter are 
both part of the problem and the solu- 
tion. 

Rob Ivry 

Several interrelated factors have con- 
tributed to the increased vulnerability 
of America’s youth. First is the rapidly 
changing labor market. Over the past 
several decades, the relatively high- 
paying, low-skill manufacturing jobs 
that provided gateway employment for 
generations of young Americans, in- 
cluding those lacking a high school di- 
ploma, have disappeared. The skill re- 
quirements for today's entry-level jobs 
are more demanding and the so-called 
"youth jobs” which do exist, generally 
in the retail industry, are often part- 
time, temporary, low-wage positions 
that do not pay benefits. This accounts 
for a widening income gap between 
young people with post-secondary de- 
grees, young people with high school 
diplomas, and high school dropouts. 

Second , the combination of the 
growth of single-parent households and 
a greater proportion of households 
where both parents work reduces the 
amount of time for parental guidance 
and support. 

Third, an increasing proportion of 
poor youth in urban areas are discon- 
nected from schools and the labor mar- 
ket and isolated from adults who work. 



.Urban schools have been resegregated 
based on "white flight” and residential 
housing patterns. The enrollment of 
students of color has grown to about 
75% in urban schools. Poor urban 
youth are physically isolated from job 
growth opportunities in the suburbs, 
and public transportation is inadequate 
to enable many young people to get to 
these jobs. These youth often know 
few people who work at steady, full- 
time jobs, and thus lack access to job 
networks that middle and upper class 
youth use to connect to jobs. 

Fourth , the recently enacted welfare 
reform legislation may place added 
strain on poor families and place added 
responsibilities on young people from 
households receiving welfare. In the 
presence of time limits on cash assis- 
tance and work requirements, young 
people may have to watch younger sib- 
lings, or drop out of school to work, in 
order to contribute to household in- 
come. Finally, the demographic trends 
reveal a forthcoming bulge in the youth 
population, with an increasing propor- 
tion from low income families. Unless 
the public commitment changes, the 
problems confronting young people in 
schools and in the labor market could 
worsen. 

Gary Walker 

The fact that it takes increasingly 
more time, energy and good luck for 
adults to make a decent living is the 
major social trend that weakens the ba- 
sic forces which support healthy youth 
development. Adults are not focusing, 
either privately or publicly, on what 
their youth are facing and what they 
should do to assist them, because their 
own commitments, stresses and worries 
have increased. This relatively recent 
economic squeeze on adults exacer- 
bates a longer cultural trend toward 
isolating age groups, and letting youth’s 
lives proceed without much interaction. 
The combination is devastating: youth 
simply do not get enough time and at- 
tention from adults, and our society 
poses ever more complex challenges for 
youth to grow up healthy and make the 
transition to adulthood successfully. 

The recent emergence of a strong 
political force which holds that almost 
every challenge and problem in an indi- 
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victual's life is individual and private 
rather than social and public in nature, 
is also a powerful obstacle to providing 
youth with the supports, direction and 
opportunities they need. Though this 
view has the potential to bring about a 
stronger sense of responsibility for 
youth at the individual, family and 
community level, for many families 
and neighborhoods in many areas of 
the country that will simply not be suf- 
ficient. Public action will be necessary 
also. This political force combines with 
a genuine concern for public budget 
deficits to further limit the possibility of 
strong public policies toward our youth. 

The increasing geographical distance 
of poor urban and rural communities 
from areas of job growth further hinders 
the ability of families and communities 
to undertake alone, without public in- 
tervention, the building of supports and 
opportunities their youth need. This 
phenomenon not only limits adults, but 
also presents to youth a formidable, 
limited and depressing view of what 
their future prospects are and how 
much the larger society cares about 
them. 

Our devotion to free markets and 
freedom — our hallmarks and strengths 
as a nation — have backfired regarding 
healthy youth development. Weapons, 
drugs — and a media devoted to the 
sensational, the negative, the purely 
commercial — these consequences of 
our freedom have helped shape an en- 
vironment which requires more and 
stronger adult and institutional interac- 
tions with youth, at the exact time that 
such supports are getting weaker. We 
are confused about what to do, since 
we do not like to limit adult or corpo- 
rate freedoms, and do not have much 
precedent for doing so, even for the 
good of our own youth. So we focus on 
exhortation, not public action. 

The fact that so many of the youth 
most affected by the above trends are 
of color, and from poor families, only 
exacerbates our confusion and unwill- 
ingness to act. These are the groups we 
are quickest to hold individually ac- 
countable for their problems and diffi- 
culties. We find it difficult to arrive at 
balanced accountability between the 
individual and the larger society. Thus 
it is hard for us to acknowledge that 



these forces and trends are having neg- 
ative effects on a larger share of all 
youth, regardless of their color or socio- 
economic status. And it is difficult for 
us to recognize that our collective social 
accountability for helping our youth 
face this world is not being addressed. 

Alan Zuckerman 

Youth are becoming even more iso- 
lated. Adolescence is lasting longer as 
the transition to work and a stable fam- 
ily structure occurs later in life. Single- 
parent families and two-wage-earner 
families often mean that young people 
lack a constructive relationship with an 
adult. This is coupled with rapid 
changes in the economy and work, 
which mean many young people are 
facing career preparation decisions 
without the counsel of a knowledgeable 
adult. 

Lacking information, feeling unwant- 
ed and seeing a future of short-term 
jobs paying poverty wages, many young 
people have little hope of getting a 
good job. Lacking hope, some young 
people drop out of school and further 
diminish their opportunities. Leaving 
school has always hurt one’s chances 
for getting a job, but the reduced in- 
vestment in "second chance” programs 
eliminates opportunities for the most at- 
risk young people. Youth do not vote 
and are out of the mainstream. Taxpay- 
er revolts have reduced funding for ed- 
ucation and increased tuition and fees. 
Low income young people have less 
chance than they had before. The ero- 
sion of opportunity for young people 
creates competition among youth and 
between groups of young people. Black 
and white, Anglo and Hispanic, urban 
and suburban, young women and 
young men are all competing for scarce 
resources — education, jobs, other sup- 
port. 

Finally, the criminal justice system is 
becoming the principal youth-serving 
agency in this country, and it creates a 
self-fulfilling prophesy. Young people 
are arrested in even larger numbers and 
more of them go to jail. In jail, they are 
offered little or no youth development 
or workforce development opportunities 
and they become apprentice criminals. 
The cycle is repeated for far too many 
youth, until they are incarcerated for 
long terms or finally find an alternative. 
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Question: What do you feel have been 
the effects of these trends in the 1 990s 
regarding 1 ) the nature of the Ameri- 
can workforce in the 1 990s, given the 
global economy in which the U.S. must 
now compete, and 2) the integration of 
American youth into that workforce ? 

David Brown 

Career opportunities for many of our 
nation’s young people (and their par- 
ents) are seemingly evaporating before 
their eyes. To remain competitive in the 
global economy and to satisfy investors, 
American businesses have slashed labor 
costs through drastic downsizing, wage 
and benefits cuts, plant relocations, 
and the hiring of contractual, part-time 
and temporary help. Each of these 
cost-cutting strategies contribute to the 
uncertainty and despair faced by Amer- 
ica’s emerging labor force, its youth. 
The well-paying and relatively secure 
low-skill jobs that enabled earlier gener- 
ations of marginally-educated young 
Americans to support families, pur- 
chase homes and raise their economic 
and social status have virtually disap- 
peared. 

The nation’s changing job structure 
has also contributed to increasing eco- 
nomic inequality based on educational 
attainment. The gap between the aver- 
age wages of the high school graduate 
and the college graduate has continued 
to grow since the mid-1970s. Public 
schools, which adequately prepared 
the vast number of young people for 
careers in the low-skilled manufactur- 
ing jobs generated by an industrial 
economy, have not kept pace with the 
increasing expectations and skill de- 
mands of the postindustrial economy. 
Secondary education in America is un- 
apologetically focused on the college- 
bound, while the remaining majority 
are too often offered little relevant skills 
or preparation that will enable them to 
successfully enter contemporary labor 
markets. The limited value of the high 
school diploma is demonstrated by the 
fact that high school dropouts with 
eleven years of schooling do not fare 
any worse than their peers who com- 
plete high school or achieve a GED but 
do not pursue postsecondary education 
or training. 
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At the recent National Education 
Summit convened by the National Gov- 
ernors’ Association and IBM in Pali- 
sades, New York, summit co-chair Lou- 
is Gestener, Jr., the CEO and Chairman 
of IBM, stated: "In an increasingly glo- 
bal economy, I am not liking oar (Amer- 
ica's) chances .” 

Indeed, we are already seeing the 
results of the collapse of our public 
schools, and all of us — states and 
businesses alike — are bearing the costs 
of failure. Many of the nation’s leaders 
concur that if the United States is to 
maintain its preeminent position in the 
world economy and in terms of the 
standard of living enjoyed by its citi- 
zens, strategic initiatives need to be 
undertaken jointly by the public and 
private sectors that will improve the 
transition from school to work and en- 
able young people to make productive 
contributions to the economy. However, 
the dominant theme of American edu- 
cation, which asserts that a four-year 
college education is the only reasonable 
track to economic self-sufficiency and 
social status, stands as a persistent bar- 
rier to developing additional career 
pathways for secondary-school stu- 
dents. 

Gary Burtless 

As a result of trends that have ad- 
versely affected less-educated labor 
force entrants, young men (and young 
women with very low levels of school- 
ing) earn substantially less than young 
people, with similar qualifications, 
earned twenty years ago. The loss in 
earnings has probably delayed the age 
at which they marry and may have de- 
layed the age at which they first have 
children. Adolescence has been ex- 
tended for many young people so that 
it now includes the late teens and early 
twenties. Young people have been 
forced to live with their parents longer, 
because the job market offers no obvi- 
ous way for them to earn enough mon- 
ey to support themselves. 

Andrew Hahn 

I would not want to hypothesize 
that the globalization of the workforce 
in the 1990s is the single most impor- 



o 




tant culprit for the problems that many 
young people experience. I do know, 
however, that recent research from 
Child Trends, as well as Rand’s Jacob 
Klerman, show that many young people 
can be identified in their twenties as 
suffering the long-term consequences 
of estrangement or disconnectedness 
begun in the latter part of high school. 
During the high school years they have 
been out of the workforce and out of 
school, and this has had rather dramat- 
ic effects a few years later. On the oth- 
er hand — and this may confuse read- 
ers — even among young people 
disconnected during the high school 
years, the really horrific outcomes expe- 
rienced among young adults in their 
twenties is concentrated in those who 
have had the most severe problems 
during the high school period. Put dif- 
ferently, many children — perhaps up 
to a third — suffer from a lack of en- 
gagement with mainstream institutions. 
But even among the one-third, a small- 
er group faces incredible trouble as ear- 
ly as the early twenties. 

I do not think the policy implications 
of these concentration effects have per- 
suaded policymakers. Age-appropriate 
targeting, prevention beginning with 
middle-school age youth, and income 
targeting are themes that attract con- 
siderable ambivalence. Targeting, many 
believe, is for racial and ethnic minori- 
ties (the “undeserving” poor?) or it pro- 
duces, in their view, stigma that dam- 
ages all programs — or perhaps 
Americans believe that targeted pro- 
grams are lower quality programs. Sim- 
ply put, universalistic programs are 
more palatable to policymakers. This is 
certainly the case with the several Con- 
gressional proposals for a new work- 
force development system for young 
Americans. To summarize, our research 
is showing that only a minority of the 
poor suffer really awful long-term ef- 
fects, yet our policies are generally 
turning away from these groups. 

Chris King 

Several effects should be mentioned: 

■ Increasing alienation and resentment. 

As youth/young adults see their par- 
ents being punished rather than re- 
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warded for their hard work and 
years of playing by the rules, it is 
becoming increasingly difficult to 
convince them that they are wanted 
and necessary in today’s economy 
and society. If the rewards are lack- 
ing for them and for their parents, 
while a smaller and smaller propor- 
tion of society is reaping incredible 
gains, rising resentment must surely 
result. 

■ Declining motivation. Similarly, these 
trends affect youth’s motivation for 
planning ahead, postponing immedi- 
ate gratification and investing in 
their futures whether through an ed- 
ucation or a series of jobs. Certainly, 
as we have seen, not all jobs require 
a college diploma, but most jobs 
should hold out the prospect of a liv- 
ing wage and for some chance of 
advancement over time. 

Gary Walker 

The effects of these trends are now 
becoming evident: more youth, espe- 
cially in poor and working class com- 
munities, are without the technical and 
social skills, and the “connections” 
necessary to make a good living. There 
is also more youth crime in those areas, 
of course, and disengagement from 
conventional values. The most vivid 
way to understand how difficult it is 
for today’s youth to find their way into 
this economy with some expectation of 
“success” is to look at what middle and 
upper class families are devoting, in 
terms of their financial resources and 
connections, to helping their youth pur- 
sue an upward economic path. Adults 
with good incomes are spending more 
money, and worrying much earlier in 
their children’s lives, to ensure their 
economic futures. They send their chil- 
dren to private schools and universities 
at great cost, and also spend a great 
deal of money to provide them with the 
recreating and other positive supports 
they need. If that is the realistic model 
of what it takes to integrate young peo- 
ple successfully into the workforce, we 
should be very sobered about the pros- 
pects for a high percentage of Ameri- 
can youth. 
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Alan Zuckerman 

Fewer young people are being inte- 
grated into the American workforce un- 
til their late twenties, when they are 
more mature and have higher level 
skills. Those lacking education are more 
often relegated to long-term low-wage 
jobs with little opportunity to enter a 
career that pays a living wage. The gap 
between young and old, rich and poor, 
employed and unemployed grows. More 
young people are poor and remain in 
the secondary labor market until their 
mid-to-late twenties. School-to-Work 
has the potential to ease the transition 
to employment for some young people, 
but the scale is not yet sufficient to 
reach its goal of “all youth." 



Question: What social policies and 
program strategies over the past decade 
or so do you feel have been the most ef- 
fective in enhancing the life chances of 
American youth, and in resolving or 
mediating youth education , training 
and employment problems — based on 
insights from competent policy analyses 
and evaluation research ? 

David Brown 

Although many of the recent evalua- 
tions of youth programs have been dis- 
couraging, many of the findings should 
not be surprising given the nature of 
the predominant narrowly-focused, 
short-term and deficit-driven approach- 
es. Research suggests that short-term 
programs that focus on improving job 
search skills or that simply offer remedi- 
ation or GED preparation at best have 
short-lived success, and they do not 
adequately prepare many youth for suc- 
cess in the labor market. However, 
some recent research has demonstrated 
that comprehensive and developmentally- 
oriented strategies can yield more posi- 
tive outcomes. 

■ Community service: research has 
demonstrated that young people 
who engage in service to their com- 
munity display higher levels of 
“prosocial development.” One study 
found that service participants expe- 
rienced significant decreases in tru- 
ancy, deviant social behavior and 
disciplinary problems. 

■ Sustained supports : research con- 
ducted by the Heller School at Bran- 
ded University of the Quantum Op- 
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portunities Program, a multi-site 
youth development demonstration 
targeting youth in AFDC families en- 
tering the ninth grade, validated the 
potential of comprehensive programs 
that provide sustained (four years 
plus) and consistent support, prod- 
ding, and encouragement. The eval- 
uation found that the participants 
achieved more positive outcomes 
than a group of comparable youth. 

■ Mentoring : a recent Public/Private 
Ventures study affirmed the efficacy 
of mentoring programs. The study 
conducted at eight local affiliates of 
Big Brothers/Big Sisters of America 
found that mentored youth were less 
likely to start using drugs and alco- 
hol; had improved school attendance 
and performance; had improved atti- 
tudes towards completing school; 
and had improved peer and family 
relationships. The researchers 
stressed that the rigid standards and 
supports employed by the national 
Big Brothers/Big Sisters office were 
critical to generating the encourag- 
ing impacts. For instance, all local 
affiliates must adhere to standards 
governing the screening and accep- 
tance of both youth and adults, the 
training of volunteers, the matching 
process, the frequency of meetings, 
and the ongoing supervision of 
matches. 

■ Contextual/ applied learning: the 
Center for Employment Training’s 
contextual learning model has been 
proven to be effective in improving 
the outcomes of out-of-school youth. 
The San Jose, California-based pro- 
gram uses a holistic approach that 
integrates basic skill remediation and 
job skill learning. CET first identifies 
jobs that are in demand and then 
builds a training curriculum focused 
on job-specific skills. Basic skill de- 
velopment is embedded in the real- 
world training curriculum. 

■ Comprehensiveness: The U.S. De- 
partment of Labor’s Youth Opportu- 
nities Unlimited demonstrations ex- 
plored the potential effectiveness of 
a holistic service approach that em- 
phasized positive youth development 
and was targeted on all youth living 
in impoverished communities. This 
demonstration, which was codified 
as Youth Fair Chance in 1992, “satu- 
rated” six urban and one rural high- 
poverty areas with job training, edu- 
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cation services, recreation and other 
services for all youth, regardless of 
their income or educational status. 

An evaluation of the original seven 
pilot sites conducted by the Acade- 
my for Educational Development 
found that in six of the seven com- 
munities the overall dropout rates 
declined by as much as fifty percent 
in one community, and in all six 
communities for which data were 
available the rate of births to teen- 
age girls also declined. 

Gary Burtless 

So far, our nation has not discovered 
ways to ameliorate the problems of 
young people who do not flourish in 
school. It has not established institu- 
tions that are parallel to college but 
that can impart occupational skills to 
youngsters who fail to prosper in the 
classroom. There may be two promising 
signs for the future, however. Commu- 
nity colleges have increasingly taken on 
the role of preparing young people for 
specific occupations. Though most two- 
year colleges probably emphasize desk 
work too much, at least they are willing 
to offer credit courses for the acquisi- 
tion of occupational skills. 

Another promising sign is the grow- 
ing recognition at the state and local 
level that the current school-to-work 
system is failing our non-college-bound 
youngsters. Recognizing a problem is 
not the same as creating a solution, but 
it is a start toward framing one. 

Andrew Hahn 

I think that our evaluation of the 
Ford-funded Quantum Opportunities 
Program (QOP) should be of keen inter- 
est to workforce development special- 
ists because it used a rigorous research 
design (random assignment along with 
case studies) to track over time the 
success and problems experienced by 
small groups of young people involved 
in an interesting program with several 
special design elements. These design 
elements included a financial incentive 
for students (stipends for program par- 
ticipation, bonus payments for level of 
effort, and an accrual account for use 
in an approved program of post-second- 
ary education); special incentives for 
staff; and a philosophy of sticking with 
kids and not thinking of them simply as 
slots. But mostly, what made QOP spe- 
cial is that it tested a long-duration (9th 
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grade through 12 grade, with some fol- 
low-up), year-round, multi-service ap- 
proach involving educational remedia- 
tion, exposure to arts and culture, work 
experience, community service, life 
skills and other services. It put the lib- 
erals’ mantra to the test, namely that 
long-term, age-appropriate services, in 
a community context run by caring 
adults, could make a difference. 

There are few programs like QOP in 
the United States. Our pilot research 
was encouraging enough in terms of 
net impacts on schooling, jobs, family 
life and other variables, that the Ford 
Foundation and the U.S. Department of 
Labor are replicating QOP in a number 
of sites this year. OICA, the communi- 
ty-based organization started by Rever- 
end Leo Sullivan, continues to provide 
technical assistance and Mathematica 
Policy Research, Inc. is the evaluator. 

Rob Ivry 

While the body of evidence on the 
effectiveness of youth programs is gen- 
erally discouraging, there have been 
some successes, and the research find- 
ings and best practice lessons provide a 
foundation on which to build. The re- 
sults tend to be more encouraging for 
in-school youth than for youth who 
have dropped out of school. Successful 
programs tend to have the following 
core elements in common: the presence 
of caring adults, financial incentives, 
hands-on training centered on work 
with a strong job development and 
placement component, and opportuni- 
ties for peer support and leadership de- 
velopment. 

These elements manifest themselves 
in different ways. Here are some exam- 
ples. Caring adults can be teachers, 
mentors, case managers, counselors or 
supervisors. An adult presence is the 
scaffolding that permits young people 
to grow, experiment, regress safely, 
take risks, develop standards and val- 
ues, build networks and transition to 
adulthood. Sustained adult contact over 
the four years of high school is the cen- 
terpiece of the Quantum Opportunity 
Program which produced large impacts 
on everything from school completion to 
pregnancy prevention. In Career Acade- 
mies, which restructure comprehensive 
high schools into schools-within-schools 
organized around a career theme, 
teachers stay with the same group of 






students throughout high school, thus 
creating a more supportive learning en- 
vironment. Adult mentors provided 
through Big Brothers/Big Sisters of 
America led to improved school atten- 
dance and reductions in substance 
abuse. 

Programs have used financial incen- 
tives in varying ways. Ohio’s LEAP pro- 
gram uses a package of financial incen- 
tives and penalties to induce teen 
parents on welfare to stay in or return 
to school. While the overall results are 
mixed, the program had positive effects 
on school attendance, high school di- 
ploma/GED rates, and employment 
rates for teens who were in-school 
when the program began. QOP used fi- 
nancial incentives tied to hours of par- 
ticipation and achievement of specified 
benchmarks. Both of these approaches 
have rewarded young people who have 
“played by the rules". 

Many successful programs feature 
paid work experience offered in combi- 
nation with education, or provide 
hands-on experiential training in occu- 
pational areas experiencing job growth. 
The Youth Incentive Entitlement Demon- 
stration, which operated in the 1970s, 
was the most ambitious youth demon- 
stration ever undertaken. Economically 
disadvantaged youth, ages 16-19, in the 
seventeen selected communities were 
guaranteed jobs (part-time during the 
school year and full-time during the 
summer) on the condition that they re- 
mained in school (or returned if they 
had dropped out) and met specified at- 
tendance and performance standards. 
The demonstration, which enrolled over 
76,000 youth, virtually eliminated the 
disparity between white youth unem- 
ployment rates and the unemployment 
rates of students of color. 

In YouthBuild, young people are 
trained in construction skills, receive 
paid work experience renovating vacant 
housing for low income families, re- 
ceive GED preparation, and participate 
in peer support and leadership develop- 
ment activities. The Center for Employ- 
ment Training's excellent track record is 
attributable to short-term training cours- 
es which integrate basic academic skills 
instruction with vocational training, a 
pervasive focus on employment, the 
use of hands-on instructional practices, 
and the involvement of employers with 
all facets of training. A recent study of 
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Youth Conservation Corps reveals posi- 
tive employment-related impacts among 
African American and Hispanic youth, 
reduced arrest rates, and a positive 
monetary benefit to society. 

These research findings and best 
practice lessons need to be disseminat- 
ed more broadly, especially to youth 
practitioners, and they need to infiltrate 
the public conscience to convert public 
skepticism into public support. Public 
policies need to be revised to encour- 
age and reward program operators for 
adopting these proven methods for 
serving young people. 

Chris King 

Those with far greater depth and ex- 
pertise in youth programs and their ef- 
fectiveness can address this question 
more fully than I. The evaluation record 
for youth interventions over the past 
twenty-five years or so is generally a 
disappointing one (Betsey, Papagoreiu 
and Hollister, 1980, and more recent re- 
views). Nor was the recent JTPA evalu- 
ation encouraging in terms of signifi- 
cant net impacts on employment and 
earnings for youth (Bloom et al., 1993/ 
1994). Many of the MDRC reports on a 
variety of youth employment and train- 
ing initiatives in recent years offer little 
to be excited about. And there is con- 
siderable debate at present as to the 
value of “job shopping” or, conversely, 
the value of a more structured transi- 
tion from school to work (Heckman et 
al, 1995; Marshall, 1995). 

Yet two strategies or programs stand 
out. First, evaluations of the Job Corps 
(not coincidentally, one of our most ex- 
pensive and intensive investments in 
hard-to-serve youth) by Mallar et al. 
(Mathematica Policy Research) and 
others have demonstrated its effective- 
ness along several key measures. Note, 
too, that even with such findings in 
hand, Congress and prior Republican 
administrations have tried hard, time 
and time again, to ‘do in’ the Job 
Corps, demonstrating that even success 
may not be sufficient to preserve fund- 
ing efforts on behalf of youth. 

Second, going even farther back, the 
Perry Preschool and Head Start evalua- 
tion findings suggest that if we are to 
make serious headway for youth, 
broadly defined, we might do well to 
concentrate even more of our resources 
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on preschool education and early child- 
hood development. By the time chil- 
dren have reached age four or five, 
many of their crucial developmental 
patterns have been well established, 
for better or worse. For far too many, 
the remainder of their education may 
be spent attempting to reverse the ef- 
fects of poor beginnings. 

Gary Walker 

Our public policies for helping youth 
develop and transition to adult roles 
have been and are generally weak. 
Public policy has always largely ignored 
adolescents and early to mid-teens. Be- 
ginning with the passage of the Job 
Gaining Partnership Act (JTPA) in the 
early 1980s, we have had no effective 
public policy for older youth who need 
assistance to enter the labor market. 

However, there have been a number 
of programmatic strategies that have 
had promising results. P/PV’s recent 
evaluation of Big Brothers/Big Sisters 
shows what can be done with early 
teens. The Quantum Opportunities Pro- 
gram in Philadelphia also indicates that 
it is possible to dramatically increase 
youths’ success and decrease their seri- 
ous problems over the course of the 
high school years. The Center for Em- 
ployment Training has had good results 
with young adults. All three of these 
programs put an emphasis on establish- 
ing strong adult/youth relationships as 
a centerpiece to their programming. All 
three focus on providing early positive 
experiences, strong feedback and ongo- 
ing support even after the initial posi- 
tive experiences. All three provide 
lengthy and consistent structure to the 
young person’s life. In short, they pay 
attention to the developmental aspects 
of youth, and do not view young peo- 
ple solely through the prism of their de- 
ficiencies. 

The hopeful aspect of these common 
features of effective programming is 
that there is nothing very mysterious 
about them. They are doable. The dis- 
couraging aspect is that they require a 
sustained commitment to young people 
as developing creatures with both 
strengths and weaknesses, whose best 
chance for avoiding serious problems 
lies as much in nurturing their 
strengths as it does in focusing on their 
weaknesses. But our approach to using 
public policy is that it is a last resort for 
fixing problems and weaknesses. And 
we want solutions quickly. 



AlanZuckerman 

We have learned a great deal in the 
past thirty years about the effectiveness 
of programs that have an impact on 
young people. Programs like School-to- 
Work, Center for Employment Training, 
YouthBuild, and Conservation Corps 
have made a significant contribution. 
However, there is no single policy or 
program model that will transform 
young people into productive, self-suffi- 
cient adults. 

NYEC has launched the Promising 
and Effective Practices Network (PEP- 
Net) to create higher standards for 
youth employment programs, based on 
the foundation of successful youth em- 
ployment/development programs, prac- 
titioner experience and research. PEP- 
Net will establish high standards for 
effective programs, invite nominations 
and recognize programs that meet these 
standards. The standards fall into three 
broad areas: youth development, work- 
force development, and quality man- 
agement. All three areas are essential 
for quality programming. Some of the 
elements of effective programs are: 

■ Education : youth must be supported 
and encouraged to stay in school, go 
back to school, or find an alternative 
school. All young people should be 
entitled to funds for their education 
until they achieve a core set of com- 
petencies. 

■ Youth are a resource: we must invest 
in the minds and bodies of young 
people. They need basic reading, 
writing, math, problem solving, in- 
terpersonal and work-related compe- 
tencies. This will require a high de- 
gree of collaboration, individualized 
plans, and involving young people in 
their own development. 

■ Community service: young people re- 
spond to opportunities to improve 
their community. Community service 
combines altruism and work experi- 
ence. 

■ Support from caring, informed 
adults: research and experience doc- 
ument the importance of a positive 
relationship with a caring adult — 
family, mentor, teacher, counselor or 
neighbor. 

■ Integration of work-based and aca- 
demic competencies: developmental 
plans that include applied learning 
combined with the development of 
basic educational competencies mo- 
tivate young people to learn by con- 
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necting acquisition of knowledge 
and competencies to work and ca- 
reers. 

■ High standards : young people and 
the organizations that serve them 
must be judged against very high 
standards of performance to assure 
that young people meet or exceed 
the highest academic and employer 
standards. 



Question: What are your recommen- 
dations for future directions in social 
policy and programming for youth , giv- 
en the problems , successful efforts and 
possibilities you have identified? 

David Brown 

Researchers and leading youth ser- 
vice practitioners alike are increasingly 
promoting and adopting an asset-driven 
approach for addressing youth prob- 
lems. They concur that most children 
and adolescents have basic develop- 
mental needs that are frequently ig- 
nored by the community institutions 
charged with serving them. To promote 
the development of comprehensive 
youth development programs, commu- 
nities should collaboratively fashion a 
continuum of developmental^ appropri- 
ate services and activities that build the 
social, moral, emotional, physical, voca- 
tional and cognitive capacities of young 
people. In particular, activities should 
be designed that promote responsibility 
and self-reliance, foster positive peer 
interactions, and ameliorate the nega- 
tive influences of adolescent peer 
groups. In addition, as job security has 
become a relic of a bygone era, young 
people must be encouraged to pursue 
lifelong learning opportunities to remain 
economically competitive. 

Principles for policymakers to con- 
sider, based on a small meeting of 
many national youth policy and pro- 
gram experts that was convened by the 
National Governors’ Association in Octo- 
ber, 1995, include the following: 

■ Provide resources to enable commu- 
nities to provide nurturing and safe 
places for young people. 

■ Target scarce resources on disadvan- 
taged communities and make servic- 
es accessible to all youth in the 
community, rather than stigmatize 
youth by labeling them as at-risk, 
basic skills deficient, teen parent, 
delinquent, etc. 
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■ Build on the strengths of youth 
(rather than focus on their perceived 
deficits), and positively channel 
their energy and enthusiasm, recog- 
nizing and exploiting the power of 
the adolescent peer group. 

■ Empower youth by involving them in 
governance and decision making and 
promote citizenship and responsibili- 
ty through community service. 

■ Coordinate or integrate the full range 
of youth services within the commu- 
nity, including recreation and health 
services. 

■ Link to the region’s economic devel- 
opment strategies, to ensure that 
youth are exposed and connected to 
real employment opportunities. 

■ Provide contextual/applied learning 
opportunities and integrate work- 
based and academic learning. 

■ Provide consistent and credible adult 
mentors and role models. 

■ Establish high standards for all 
youth and incentives to encourage 
the achievement of competencies 
and goals. 

■ Provide sustained services and sup- 
ports. 

Gary Burtless 

Over the next decade, the main goal 
for sensible policy is to develop effec- 
tive program models to impart skills to 
the young people who will not enter or 
complete college. Competent evaluators 
should attempt to learn whether certain 
school-to-work program models yield 
better results than others. In the long 
run, this kind of information will have 
enormous value to policymakers and to 
young people trying to get established 
in the job market. At the moment we 
know very little about the long-term 
benefits of a reformed school-to-work 
system. Students, employers, and 
school administrators have too little ex- 
perience under recently developed pro- 
grams for us to draw reliable conclu- 
sions about the longer-term effects of 
mature programs that are run by expe- 
rienced educators and employers. How- 
ever, careful analysis of results from a 
variety of good programs can shed light 
on whether certain program models 
show special promise. 

The Office of Technology Assessment 
has identified six broad models of work- 
place-centered learning — youth ap- 



prenticeship, classroom training, cooper- 
ative education, school-to-apprenticeship 
training, school-based enterprises, and 
career academies. Within each model, 
schools and employers may take differ- 
ent approaches toward training and re- 
warding participants for the work they 
do. Some programs emphasize the in- 
volvement of adult mentors for each 
trainee. Others emphasize classroom in- 
struction in schools. Others may require 
students to spend a large proportion of 
the academic year in a job — say, fif- 
teen or more hours a week — while 
other programs offer no pay to trainees 
who work. Experience has shown that 
one of the toughest challenges facing 
workplace-centered learning is recruit- 
ment and retention of local employers 
to participate in school-to-work pro- 
grams. Some school systems may devel- 
op innovative methods to attract and 
keep employer participation. 

The variety of program models that 
school systems and localities have de- 
veloped opens rich possibilities for a 
sound evaluation of alternative ap- 
proaches. Investigators can attempt to 
determine whether one program model 
has been consistently more successful 
than others. They can try to estimate 
the relative effectiveness of small varia- 
tions in treatments within the same ba- 
sic model. At the moment, analysts and 
policymakers have little reliable infor- 
mation to show whether one approach 
to workplace-based education is better 
than another. Since many local pro- 
grams are now in their infancy, pro- 
gram administrators have to learn about 
the relative effectiveness of different 
approaches. For that reason, there is 
likely to be a large payoff from research 
that reliably distinguishes between suc- 
cessful and unsuccessful approaches, 
as well as from research that shows 
whether certain kinds of students and 
dropouts can benefit from particular 
school-to-work models. An important 
focus of a good evaluation should be 
measurement of the relative effective- 
ness of different program models. 

Andrew Hahn 

Research supports the recommenda- 
tion to focus on youth as youth, some- 
thing that was often lost as the work- 
force field rushed to create specialized 
niches. So, for example, resume writing 
workshops may not be appropriate for 
fourteen-year-olds, but higher educa- 
tion awareness programming is terribly 
important for this age group. 



Research also leads to the impor- 
tance of strengthening the connections 
between school, basic skills, work and 
the workplace. For this reason, I fully 
endorse the goals of the national 
School-to-Work Opportunities Act, but 
I also have no illusions about its effec- 
tiveness as an anti-poverty strategy. For 
that we have to think more imagina- 
tively about involving community 
groups in school-based workforce de- 
velopment, helping these groups to de- 
velop their capacity, integrating "hard 
and soft” skills training, and targeting 
national workforce policies to either 
neighborhoods or people who need the 
help most. 

Research also recommends the im- 
portance of increasing the intensity of 
services. This is why I addressed the 
importance of the Quantum Opportuni- 
ties Program in a preceding question, 
and that is why I am enthusiastic 
about the work of the International 
Center for Residential Education, a 
Washington DC-based group that is try- 
ing to help communities establish resi- 
dential education options that avoid the 
stigma of the institutional settings of 
the past. 

I also believe that policymakers and 
community planners need to think more 
about managed care tools that can 
track clients through an array of servic- 
es in the community. And I would like 
to see new linkages between communi- 
ty development and school-to-work ini- 
tiatives, in order to promote longer- 
term, more intensive services. I think 
intensity could be promoted by more 
use of youth entrepreneurship pro- 
grams. These combine traditional class- 
room training with hands-on experien- 
tial learning through business 
development. Students stay in contact 
with these programs longer than might 
be the case with regular youth pro- 
grams, because the students need ad- 
vice and assistance from the program 
staff to help maintain the small busi- 
ness enterprises. 

Finally, research suggests that we 
must promote effectiveness in what 
will be a decentralized structure. This 
translates into local monitoring and 
evaluation of workforce development 
programs, more capacity building, and 
a greater role for university-community 
partnerships. It also speaks to involving 
all sectors of the community in work- 
force development activities, including 
employer groups — although we have 
few good models of employer-side inter- 
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ventions with implications for deep ur- 
ban poverty. 

Rob Ivry 

We need a national policy of ade- 
quate and sustained investment in 
young people. The negative youth re- 
sults from the national evaluation of the 
Job Training Partnership Act reveals the 
flawed strategy of trying to shoehorn 
youth into an adult system, with per- 
formance standards that work at cross- 
purposes to the developmental needs of 
young people. The federal investment 
in youth programs has been cut back in 
recent years, and the eventual passage 
of a workforce development block grant 
could further infringe on youth funding 
and services. Public policy needs to 
recognize the wisdom of a youth in- 
vestment policy with the underlying 
premise of “building on strengths” rath- 
er than "remedying deficits” and which 
incorporates the following principles: 

■ Implements a funding structure that 
targets funds where they are needed 
most and provides strong financial 
incentives for states and localities to 
adopt and tailor proven and promis- 
ing program approaches such as 
those described above. 

■ Adopts a flexible system of perfor- 
mance standards which acknowl- 
edges that the youth maturation pro- 
cess varies by individual, and 
rewards progress toward education- 
al, employment and personal devel- 
opment outcomes. 

■ Strengthens families and communi- 
ties to provide greater support to 
youth and increases the engagement 
of young people in productive activi- 
ties after school hours. 

■ Authorizes career preparation and 
paid work experience as fundamen- 
tal activities within a youth develop- 
ment strategy in conjunction with 
education, leadership development, 
and skill-building activities. 

■ Extends the core elements of the 
school-to-work movement — school- 
based learning, work-based learning, 
and activities which connect the 
two — to an overall youth develop- 
ment strategy. 

■ Supports creative funding strategies 
for financing alternative education 
and employment programs for out- 
of-school youth by, for example, 
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making "average daily attendance” 
funds portable (the funding follows 
the student) and by broadening the 
use of criminal justice funding for 
preventative activities. 

■ Continues to encourage summer ed- 
ucational activities to be offered in 
conjunction with summer employ- 
ment. 

■ Provides strong support for new am- 
bitious demonstrations to expand the 
knowledge base concerning what 
works for youth — for example, 
demonstrations which challenge the 
existing paradigm of education as a 
prerequisite for employment, 
through alternative strategies which 
provide skill building and education 
activities on the back end of em- 
ployment once students start work- 
ing. 

There also needs to be a recognition 
that solving the youth problem and ex- 
panding opportunities for young people 
will not come cheaply and the results 
will not occur overnight. There needs to 
be sustained investment and public ex- 
pectations need to be realistic about 
what can be achieved in the short- 
term. The future health and vitality of 
our country hinges on the extent to 
which we adopt the policy tools to en- 
able the current generation of young 
people to succeed in school and the la- 
bor market, and successfully make the 
transition to adulthood. 

Chris King 

I will offer a number of broad social 
policy and specific program recommen- 
dations, beginning with the former. 

First, we truly must begin to view ah 
children as our nation’s future and 
bring our social policies around accord- 
ingly. We cannot afford to ‘write off’ 
any child. In crass terms, we all need 
them to help support us — physically, 
emotionally and financially — - in our 
dotage! More importantly, it is a moral 
imperative, especially in a nation with 
the wealth we possess. We cannot af- 
ford to punish children for the per- 
ceived ill behavior of their parents by 
consigning them to another generation 
of poverty and insecurity. Each youth 
in this country must be viewed as a 
valuable asset not to be squandered. I 
am reminded that when our attempts 
to launch satellites into space failed 
miserably in the late 1950s, our re- 



sponse was not resignation but rather a 
redoubling of our efforts until we be- 
came the first to put a man on the 
moon in 1969, only a bit more than a 
decade later. We need to approach our 
experience with youth initiatives in a 
similar fashion. 

Second, we must begin to restore 
the employer/employee compact that 
existed in this country for most of the 
period from the late 1930s to the 1980s. 
This compact was never as strong as it 
was in our trading partners in Western 
Europe or Japan, but it was essential to 
the fabric of the nation and our market- 
based system. The current trend, in 
which the community of employers 
views its primary and possibly its sole 
obligation as being to its shareholders 
to the exclusion of its employees — 
whether concerning the provision of 
equitable wages, health and related 
benefits associated with employment, 
and some sense of security — must be 
reversed. 

Third, we should contemplate 
adopting capital budgeting nationally 
and in the states, encompassing both 
human capital as well as infrastructure. 
Why should the financing for education 
and training, which Dennison (1962, 
1991), Harbeson (1973) and others 
have demonstrated to be the dominant 
factor in explaining our nation’s eco- 
nomic growth over time, be handled on 
a current accounts basis just as the 
staffing of prisons or income mainte- 
nance, when it is more appropriately 
viewed and financed as an investment? 
It is also time to address the fact that 
we have increasingly pushed the bur- 
den of financing postsecondary educa- 
tion towards individuals and families, 
while the tax code offers them little 
support. We must open the door to 
some form of individual training ac- 
counts or related measures with prefer- 
ential tax treatment. 

More specifically, there are several 
other actions we should pursue: 

■ The federal and state governments 
should encourage employers to pur- 
sue modes of high performance work 
organization (e.g., team manage- 
ment, shifting work responsibilities 
to ever lower levels, pay based in 
part on group effort, flexible produc- 
tion, the use of the latest technolo- 
gies — Marshall, 1992; Batt and Ap- 
plebaum, 1994; Osterman, 1995). 
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This should include the adoption of 
one of the centerpieces of the ISO 
9000 process, namely fostering tight 
and lasting producer/supplier rela- 
tionships, in this instance those be- 
tween employers and schools as sup- 
pliers. To become ISO-certified, 
which is increasingly a prerequisite 
for conducting business in Western 
Europe and elsewhere around the 
globe, a firm must demonstrate that 
it has cultivated such supplier rela- 
tionships and that it ensures that 
these suppliers are trained to the 
same standards and expectations. 

Were employers to view schools as 
one of its primary suppliers, not just 
becoming involved but rather truly 
engaging schools as an integral part 
of their production/service process, 
youth would face far better and 
more structured employment oppor- 
tunities. 

■ The Commission on the Skills of 
America's Workforce's America's 
Choice report (1990) contained a 
number of recommendations on 
youth policy which should be seri- 
ously considered for adoption. 

Among these are use of the Certifi- 
cate of Initial Mastery and others. 

■ Nationally, Congress and the Admin- 
istration should pursue full and con- 
tinuing funding of the School-to- 
Work Opportunities Act of 1994. As 
things now stand, the federal gov- 
ernment will provide several years of 
"seed money” after which time it is 
up to financially-strapped states and 
local governments to pick up the tab 
and continue the work. This is far 
too important an effort for the nation 
as a whole to let it simply devolve to 
subnational levels of government. As 
we have seen too often, some states 
will do well with this responsibility 
while others are likely to fail misera- 
bly. We cannot risk this outcome. 

■ Finally, although it may be beyond 
the task at hand, rather than ad- 
dressing child care as a support ser- 
vices issue in our many E&T pro- 
grams, we must see it as yet another 
opportunity for investment. Too 
many states, not to mention the 
Congress, see funding for child care 
for welfare caretakers. Food Stamp 
recipients or job training participants 
solely as a cost to be avoided, when 
in fact it is an opportunity to provide 
high quality, early childhood devel- 
opment services for these children. 
This applies also to our work- family 







policies in the U.S. Note that several 
evaluations have shown that partici- 
pants in these programs see access- 
ing quality child care as one of the 
major benefits of participation (King 
et al., 1994; MDRC, 1996). 

Gary Walker 

The first thing we will have to do is 
to accept public policy as a tool for 
improving the basic supports and op- 
portunities that youth need for healthy 
development, and not see it as a neces- 
sary evil that only gets used when 
problems get out of hand. The recent 
School-to-Work Opportunities Act, and 
the proposed Youth Development Block 
Grant, are good signs that some policy- 
makers are ready to make that shift. 

The second necessary change is that 
we must accept that it takes resources 
and time — in short, genuine public 
commitment — to accomplish sustained 
and durable positive development in 
young people. Neither Big Brothers/Big 
Sisters nor CET developed its organiza- 
tional strengths overnight. The need for 
patience in building organizational ca- 
pacity is especially important in helping 
youth connect with the labor market, 
since that requires developing good 
working relationships with employers. 

The good news is that the program 
strategies that have been effective 
could be taken to scale without deci- 
mating the federal budget. For exam- 
ple, our estimate is that a $3.0 to $4.0 
billion commitment of public resources 
would provide intensive mentoring for 
all young adolescents who need it. Of 
course, it would take several years to 
build the institutional capacity to spend 
that money effectively. 

Assuming we can achieve the 
change in approach and increase in re- 
sources and time commitment neces- 
sary, we could speed the process of 
achieving widespread effectiveness if 
we would drop the tired polarity of 
"central cookie cutter” vs. “all truth is 
local” and administer and monitor pub- 
lic funds on the more realistic basis 
that "both ideas are true.” There are 
core principles of effective program- 
ming that must be adhered to in every 
situation — and each locality does have 
different conditions and opportunities 
within which to carry out those core 
principles. In short, there are uniform 
elements to effective programming, and 
still plenty of room for local flexibility. 

To administer public policy in this 



framework requires a bit more hard 
thinking and a lot less rhetoric than is 
typical for the formation and execution 
of public policy. 

Alan Zuckerman 

I would propose the following rec- 
ommendations: 

■ Maintain a federal commitment to at 
risk youth programs'. There must be 
adequate funding for both summer 
and year-round programs. Rational 
discussion about the merits of in- 
vesting in youth is vital to the future 
of this country. 

■ Focus on special needs of urban and 
rural youth: Our cities cannot be 
forgotten and abandoned, and 
young people must be connected to 
education and jobs. There must be a 
revitalized urban policy. Funding 
must be targeted to cities and poor 
rural areas where the needs are 
greatest. 

■ Accountability: Demand that youth 
programs meet rigorous standards. 
Young people must be prepared to 
meet world-class standards. 

■ Invest in building the capacity of 
community-based programs and staff: 
The uncertainty of federal funding 
and the transition to "block grants” 
is placing many experienced and 
proven community-based organiza- 
tions in jeopardy. Quality services re- 
quire skilled staff and stable organi- 
zations. 

■ Jobs for at risk youth: Young people 
enroll in job training programs be- 
cause they want to get a job, earn 
money and support themselves. Of- 
ten entry-level jobs are scarce, pay 
little money, and may be dead-end. 
Central cities and remote urban ar- 
eas have more job seekers than jobs. 
Job creation, including publicly-sup- 
ported jobs, and the development of 
career pathways are essential if at- 
risk young people are to find jobs 
that enable them to support them- 
selves. 

■ Young people are a resource , not a 
problem : Investments in youth em- 
ployment and youth development 
are a more effective and less costly 
way to reduce crime and drug 
abuse. 
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There is much food for thought in 
the responses of the “expert panel.” Its 
members include colleagues whose 
work I know and respect. Many have 
helped define the field in which we all 
operate, mapping out the overlapping 
boundaries between youth develop- 
ment, education and vocational/occu- 
pational training. 

Especially when it comes to old 
friends, I balk at trying to reduce di- 
verse, collective insights so that they fit 
into a nutshell. Rather than having the 
last word, let me suggest implications 
and play off both what was said and 
what was left unsaid. 

As I read their social diagnoses, the 
panelists take a grim view of the cur- 
rent environment within which youth 
come of age and are integrated into the 
workforce. Words like frustration , bleak- 
ness, insecurity , despair ; and resentment 
recur as a descriptive motif for the pre- 
vailing mood. For more and more Amer- 
icans, the opportunity structure is col- 
lapsing. Access and participation in 
effective education and training pro- 
grams are limited to the most prepared. 
De-industrialization has cut the rungs 
out of the ladder to a middle class life 
for most Americans. Global forces are 
causing the labor market to buckle and 
shift. Demands increase for high-end 
skill sets, thereby widening the gap be- 
tween education haves and have nots. 
The knowledge industry and informa- 
tion age promise no end to these high- 
end demands. 

In this rapidly evolving two-tiered 
society, winners take all, and losers be- 
come objects of indifference or aver- 
sion, viewed either as Darwinian misfits 
or as moral pariahs who have no one 
but themselves to blame. The rise of 
the libertarian right has helped create a 
politics where every solution is a pri- 
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vate solution, where the notion of a 
commonwealth and a public interest 
has dimmed to a faint glimmer. 

It seemed to me that what the pan- 
elists are depicting is the emergence of 
a system of neo-apartheid. This really is 
the substance of their findings: a grow- 
ing division between us and them , a 
cordoning off by geography , class and 
race of those groups whose thinking , be- 
havior , and performance earn them out- 
cast status, and even more so when 
these groups challenge majority stan- 
dards for their well being and insist on 
having a say. But there seems to me a 
curious disparity between the central 
diagnosis, which is grave, and pro- 
posed treatments, which are mild. 

The warning signs are as plain as 
the handwriting on the wall. Yet the 
scope and depth of our policies and 
program recommendations appear 
hopelessly incommensurate with the 
underlying problems they are intended 
to address. The principles and ap- 
proaches which panelists promote are 
fine as far as they go. There is genuine 
worth in “asset-driven” strategies; in 
flexible, developmental^ appropriate 
performance standards; in a deeper 
knowledge base; and in the intensified 
service delivery embodied in models 
like QOP. But again, I am struck by the 
disconnect between the modesty of 
these proposals and the sense of loom- 
ing crisis that many of us feel. 

I think many of us are searching for 
ways to reduce this disconnect. It is 
hard going. The culture into which we 
have been socialized places a premium 
on analytic detachment. We are profes- 
sional outsiders, trained and rewarded 
to observe, measure and report on 
what others are doing to bring about 
social or institutional improvement. As 
individual citizens we may choose to 
become engaged as advocates and ac- 
tivists. But as “experts” in the field we 
are expected to keep at a safe remove 
from the subjects of our inquiry. We are 
taught that distance protects the credi- 
bility of our findings, and hence the in- 
fluence that we might bring to bear on 
the policy process. 

There was a time when we proceed- 
ed on the assumption that research and 
demonstration activities would help 
federal policymakers make wise choic- 
es. But devolution has shifted the cen- 
ter of political gravity from the national 
to the local. This change has enormous 
implications. In the current environ- 



ment, policies will take shape from the 
bottom up. Space will increase for com- 
munities to experiment, to agitate and 
to mobilize. All of this presents new 
challenges and new opportunities for 
youth research. 

The challenge is to revise our role 
and expand our repertoire so that we 
can be more effective and trusted allies 
with community change agents. This 
does not mean signing on as booster, or 
abandoning our commitment to analytic 
rigor. It does mean learning to negotiate 
relations which are dialogic and recip- 
rocal. It does mean respecting different 
voices, different ways of knowing. It 
does mean demystifying and democra- 
tizing, and disseminating expertise. 

The opportunity is to break down 
the intellectual apartheid that has long 
separated the academy from the com- 
munity. We see a society more and 
more fragmented and polarized. The 
collaborative content of our work 
should reflect our dedication to a 
wholesome and egalitarian polis. We 
should practice what we preach. 



Editor’s Note 



An emphasis on young adults’ need 
for education, skills, training and em- 
ployment opportunities is a consistent 
message in the panel’s responses. An- 
drew Sum’s research contributes much 
to that perspective. His thinking, based 
on extensive research, is revealed 
clearly in “Young Workers, Young Fami- 
lies, and Child Poverty,” a chapter by 
Sum, Clifford Johnson, and Neal Fogg 
in Garth and Stephen Mangum’s new 
edited book, Of Heart and Mind: Social 
Policy Essays in Honor of Sar A. Levi- 
tan . Sum and his colleagues define the 
youth problems demanding change, 
identify the interrelated and complex 
causes of these problems, and suggest 
change strategies. 

In Sum’s view, a significant propor- 
tion of American youth are experienc- 
ing severe labor market problems due 
to a lack of education, training, and 
work experience, jeopardizing their 
own futures and the future of the Amer- 
ican economy. He points to a wealth of 
data pointing to the sharp decline over 
the past two decades in the inflation- 
adjusted weekly and annual earnings 
of young adults, despite renewed job 
growth. Youth, he claims, have been 
adversely affected by the transforma- 
tion of the U.S. wage structure — with 
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resulting difficulties in securing full- 
time, year-round employment, and with 
the growing incidence of child poverty 
Such trends, he contends, have seri- 
ous negative consequences for youth 
and society Sum’s recommendations 
for change fall heavily on education, 
skills training, and the expansion and 
diversification of job opportunities. 

For more detail about the thinking of 
the members of the expert panel, 
please see the following references. 

Publications in the 1980s 

Burtless, G.T. ‘The Impact of Welfare Re- 
form on Recipient Employment, Earn- 
ings, and Income," in Welfare Policy for 
the 1990s edited by D. Ellwood and R H. 
Cottingham. Cambridge, MA: Harvard 
University Press, 1989. 

. “Work Programs in Welfare 

and the Difference They Make,” in Re- 
forming Welfare: Lessons, Limits, and 
Choices edited by R. Coughlin. Albu- 
querque, NM: University of Mexico 
Press, 1989. 

Charner, I. (With P. Barton et al.) Worklife 
Transitions: The Learning Connection. 
McGraw-Hill, 1982. 

. (With B. Fraser) Youth and 

Work: What We Know, What We Don’t 
Know, What We Need to Know. Wash- 
ington, D.C.: William T. Grant Founda- 
tion’s Commission on Work, Family and 
Citizenship, 1987. 

. "Employability Credentials: A 

Key to Successful Youth Transition to 
work,” in Journal of Career Develop- 
ment, 15, 1988. 

Hahn, A. B. (with J. Danzberger) Dropouts 
in America. Washington, D.C.: Institute 
for Educational Leadership, 1989. 

. (with R. Lerman). What Works 

in Youth Employment Policy? How to 
Help Young Workers from Poor Families. 
Washington, D.C.: National Planning As- 
sociation, 1985. 

. (with R. Sheets, R. Lerman 

and E. Butler) Who Will Train and Edu- 
cate Tomorrow’s Workers? The Financ- 
ing of Non-College Bound Young Work- 
ers’ Recurrent Education. Commission 
on Work, Family and Citizenship. Wash- 
ington, D.C.: W. T. Grant Foundation, 
1988. 

. “Job Creation Policies: Do 

They Stimulate Demand for Disadvan- 
taged Young Workers?” in Portfolio: In- 
ternational Economic Perspectives. 
Washington, D.C.: U.S. Information 
Agency, 1987. 



. (with M. Forcier) “The Impact 

of Employment and Training Programs 
on Work Attitudes of Disadvantaged 
Youth: A Synthesis of Theory and Evi- 
dence,” in Youth Employability: Five 
Award Winning Monographs. National 
Center for Research on Vocational Edu- 
cation. Columbus, OH: Ohio State Uni- 
versity, 1982. 

Ivry, R. (with Alan Weisberg) The 22% Solu- 
tion: LEA/CETA in Youth Employment and 
Training Programs. Washington, DC: 

U.S. Office of Education, 1980. 

. (with Carl Wolfhagen) 

Strengthening the School-to-Work Link- 
age in Youth Incentive Entitlement Pilot 
Projects. New York, NY: MDRC, 1982. 

King, C. T. (with V. Geraci) Employment and 
Training Program Performance : Long- 
Term Earnings Effects and Short-Term 
Indicators. Working Paper. Center for the 
Study of Human Resources. Austin, TX: 
University of Texas, 1981. 

and D. T. Schexnayder. Wel- 
fare Dynamics in Texas: An Exploratory 
Analysis of AFDC Turnover and Pro- 
gram Participation. Austin, TX: Center for 
the Study of Human Resources, 1988. 

. (editor) Coordinating ETA-Ad - 

ministered Programs: Issues and Rec- 
ommendations. Washington, D.C.: U.S. 
Department of Labor, 1989. 

Zuckerman, A. Establishing a Cooperative, 
Interactive Employment and Training 
Network. Paper for Florida Employment 
and Training Association, 1984. 

. “Two Decades and Two Deca- 
logues of Employment and Training,” in 
ADHERENT, Volume 4, Winter 1983. 

. Individualized Service Deliv- 
ery. Paper for National Governors’ Asso- 
ciation, 1985. 

. At Your Fingertips. Paper for 

National Commission for Employment 
Policy, 1985. 

Publications in the 1990s 

Brown, David E. “The Case for Second 
Chance Programs for Out-of-School 
Youth,” in Labor Notes, NGA, October 
1992. 

. A New Paradigm for Youth 

Employment and Training Programs, Na- 
tional Governors Association, 1993. 

. The Role of Job Training Part- 
nership Act Programs in School-to-Work 
Transition, NGA, 1994. 



BEST COPY AVAILABLE 44 

o 

ERIC 



. Human Resource Investment 

Councils: Summary of State Efforts, 

NGA 1994. 

Burtless, G.T. “Current Proposals for 

School Reform: An Economist’s Assess- 
ment,” in Economic Consequences of 
American Education edited by A. P. 
O’Brien and R. J. Thornton. Greenwich, 
CT: JAI Press, 1993. 

, M.N. Baily and R. E. Litan. 

Growth with Equity: Economic Policy- 
making for the Next Century Washing- 
ton, D.C.:The Brookings Institution, 

1993. 

. “Meeting the Skill Demands 

of the New Economy,” in Labor Markets, 
Employment Policy, and Job Creation 
edited by L. C. Solmon and A. R. Leve- 
son. San Francisco, CA: Westview 
Press, 1994. 

. “Rising Wage Inequality and 

the Future of Work in America,” in Wid- 
ening Earnings Inequality edited by J. 
Norwood. Washington, D.C.: Urban Insti- 
tute Press, 1994. 

Charner, I. School Reform and Youth Transi- 
tion. Washington, D.C.: Academy for Ed- 
ucational Development, 1993. 

Curnan, S. P. (with Policy Research Associ- 
ates) Evaluation of the Educational 
Component of the Summer Youth Em- 
ployment and Training Program. Wash- 
ington, D.C.: U. S. Department of Labor, 
1995. 

In Support of the Youth Devel- 
opment and School-to-Career Move- 
ments and Blurring the Lines Between 
What Is Work and What Is Learning. Pa- 
per commissioned as part of the Nation- 
al Youth Research Project. Washington, 
D.C: U.S. Department of Labor, 1995. 

. Youth Fair Chance: A Working 

Model for a New Way of Thinking and A 
New Way of Doing. Conference paper. 
Washington, DC: National Convening of 
Youth Fair Chance Demonstration, 1995. 

(with T. Melchior) Moving To- 
ward Quality Programs: Defining Criteria 
for Quality Program Designs Based on 
Lessons from Research and Experi- 
ence. Commissioned report on behalf of 
U.S. Department of Labor. Washington, 
D.C.: National Youth Employment Coali- 
tion, 1994. 

(with H. Bruno). Measuring the 

Effectiveness of Workforce Preparation 
Policies at the State Level. Commis- 
sioned paper. Washington, D.C.: National 
Governors’ Association, 1994. 
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Charting a Course for Excel- 
lence in Youth Development, Education 
and Employment Programs. Commis- 
sioned report. Princeton, NJ: Educational 
Testing Service, 1994. 

Hahn, A. B. America's Middle Child: Making 
Age Count in the Development of a Na- 
tional Youth Policy. Center for Human Re- 
sources. Waltham, MA: Brandeis Univer- 
sity, 1995. 

. “National Human Resource 

Policy for Disadvantaged Youth” in Di- 
lemmas in Youth Employment and Train- 
ing Policy: Lessons from the Youth Re- 
search and Technical Assistance 
Project - Volume 2. Washington, D.C.: 
U.S. Department of Labor, 1995. 

. “Toward a National Youth De- 
velopment Policy for Young African 
American Males in Nurturing Young 
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Editor’s Introduction 



One of the major tasks of societies 
is the socialization of its youth--that is, 
the teaching of youth to play responsi- 
ble, meaningful, satisfying adult roles, 
however those roles are defined by a 
particular society. Western civilization 
and the U.S. society in the 1990s are 
experiencing new problems in the so- 
cialization process, such as a high lev- 
el of drug abuse and crime, and the 
adoption of alternative values and 
roles whose definitions vary from the 
central tendency shaping notions of re- 
sponsibility. Increasing employment 
challenges, growing wealth and in- 
come inequality, and rising poverty in 
Western societies have limited the life 
chances of youth, complicating critical 
socialization tasks. Our explanation of 
“adulthood” is less clear. While this 
new flexibility gives greater discretion 
to youth in making decisions about 
themselves and others, and about their 
own life directions, it is emerging at a 
high price in terms of personal and so- 
cietal safety. 



This interview focuses on this new 
context for youth, particularly the envi- 
ronment in which poor youth in U.S. cit- 
ies are growing to adulthood in the 
1990s. The discussion is informed 
uniquely by a comparative analysis 
across several European societies, 
based on William Julius Wilson’s chap- 
ter in his new edited book, Poverty, Ine- 
quality and the Future of Social Policy, 
published in 1995 by the Russell Sage 
Foundation. 

William Julius Wilson is currently 
the Malcolm Wiener Professor of Social 
Policy at the John F. Kennedy School of 
Government at Harvard University. He 
was formerly Lucy Flower University 
Professor of Sociology and Public Poli- 
cy at the University of Chicago. Wilson 
is an elected member of the National 
Academy of Sciences and the Ameri- 
can Philosophical Society. He is also a 
Fellow of the American Academy of 
Arts and Sciences and the American 
Association for the Advancement of 
Science. 

Directing most of his work to the 
study of race, class and poverty, he has 
written several major books and nu- 
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merous articles that have captured the 
serious attention of both scholars and 
public policymakers. He has proposed 
a research-based theory of the social 
transformations which have occurred in 
the inner city, and has recommended a 
comprehensive policy agenda ad- 
dressing broad social problems which 
disproportionately affect the inner city 
poor. 

His book titled The Declining Signifi- 
cance of Race: Blacks and Changing 
American Institutions received the 
1980 Sydney S. Spivak Award. His 
1988 book, The Truly Disadvantaged: 
The Inner City, the Underclass, and 
Public Policy received the C. Wright 
Mills Award and was selected by the 
editors of the New York Times Book Re- 
view as one of the sixteen best books 
published in 1987. His latest book is 
When Work Disappears: The World of 
the New Urban Poor, published in 1996 
by Alfred A. Knopf. 

He is the recipient of the 1987 Mac- 
Arthur Prize and the 1988 Godkin Lec- 
tureship at Harvard, and is past presi- 
dent of the American Sociological 
Association. 
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Question: What led you to produce a 
new book on poverty and inequality? 

Response: We found the nature of 
the 1980s’ American social policy de- 
bates concerning poverty frustrating 
and disturbing. They seemed disjoint- 
ed, some positions emphasizing the im- 
pact of economic change, others giving 
primary attention to moral decline. 
Meanwhile, structural factors such as 
income and wealth inequality, rising 
worker displacement and joblessness, 
and a growing concentration of poverty 
in urban areas were having significant 
effects on life chances. We felt that a 
comparative analysis across industrial- 
ized countries would focus attention on 
these institutional factors and their im- 
plications for poverty and inequality. 

The Rockefeller Foundation provided 
the initial funding for this policy study, 
and several other foundations subse- 
quently contributed to the project, in- 
cluding the Russell Sage Foundation. 
This permitted important meetings of 
the project’s advisory committee and 
the international group of experts who 
co-authored the book. We hoped that 
the book would change the character 
of the policy debate, away from a pre- 
occupation with individual attitudes 
and behavior and toward an analysis of 
the economic context influencing the 
nature of poverty in contemporary soci- 
ety. 

Question: The economic and labor 
market influences that have condi- 
tioned poverty increasingly in Western- 
societies have been of particular inter- 
est to you . What insights does this new 
look at European poverty provide in 
this respect? 

Response: Since the late 1970s, Euro- 
pean governments have faced the re- 
turn of mass unemployment and a 
growing fragmentation of their labor 
markets. Increasing numbers of indige- 
nous workers and immigrants have fall- 
en into the secondary labor market — 
characterized by low skills, poor 
working conditions, weak union protec- 
tion, and a high risk of unemployment. 
European unemployment rates are now 
close to ten percent. At the same time 
there has been a growth in long-term 
unemployment, forming a substantial 
subgroup of individuals with loose at- 
tachment to the labor force (including 
many young people). In fact, in 1993 
half of all unemployment in Western 
Europe was long-term, compared with 
10-20% in the rest of the industrialized 
world. This subgroup included many 
young people. 



In Germany, foreign workers and 
their families were much less likely 
than native Germans to be "officially 
poor.” Subsequently, however, and par- 
ticularly in the 1990s, this changed dra- 
matically. During 1991, almost 15% of 
all foreigners utilized public assistance 
at some point; only 5% of the German 
population needed to depend upon it. 
Also, foreigners have been dispropor- 
tionately affected by the cuts in social 
benefits — that is, in means-tested pov- 
erty programs — - in which foreign work- 
ers and their families have tended now 
to be overrepresented. In the United 
Kingdom, the Thatcher government in- 
troduced major cost-cutting reforms in 
the context of 8 million people (one in 
7) of the total British population living 
in households dependent on a national 
means-tested and last-resort income 
support program. This is a significant 
increase over 4 million in 1973 and 1.2 
million in 1950. 

Worsening economic conditions in 
Europe have led to an urban housing 
crisis and insecurities in the labor mar- 
ket, as well as a perception that the 
growth of the minority and immigrant 
populations has been the major cause. 
The recessions have also exposed the 
vulnerability of ethnic communities to 
economic stagnation, industrial restruc- 
turing and the decline of traditional 
manufacturing. Related to these devel- 
opments, immigrant unemployment 
rates have been reaching 25-50% in cit- 
ies experiencing high levels of employ- 
ment for the general population. Many 
of the immigrants most acutely affected 
by recession have been second and 
third generation minority youth with 
educational deficits, who are having 
difficulty finding employment and are 
isolated and alienated from the societ- 
ies of the older generations. But these 
conditions for immigrant youth primari- 
ly reflect deteriorating economic condi- 
tions and declining organizational re- 
sources in the poorer working-class 
communities. 

However, Europe and the U.S. are in 
marked contrast in terms of urban ine- 
quality. No European city has experi- 
enced the level of concentrated poverty , 
and racial and ethnic segregation, that 
characterizes American metropolitan ar- 
eas . No European urban area incorpo- 
rates areas as physically, socially and 
culturally isolated, deteriorated or vio- 
lence-prone as American inner city ghet- 
tos. However, many European inner- 
city communities and suburban public 
housing areas have been cut off from 



mainstream labor market institutions 
and informal job networks. This 
spawns a cycle of weak labor force at- 
tachment, increasing social exclusion, 
and rising group tensions. Trends in a 
number of European countries suggest 
the beginning of urban social polariza- 
tion, leading to a concentration of social 
problems in certain areas of cities. Eu- 
ropean research shows that as poverty 
becomes more concentrated it involves 
multiple kinds of deprivation, which 
then translates to both material decay 
and reduced life quality. This cycle and 
these trends have important conse- 
quences for the socialization of the 
next generation. 

Question: What are some of the social 
policy differences between Europe and 
the U.S . that have mediated against 
sharp socio-cultural divisions within 
European cities, and the concentration 
of poverty in particular parts of these 
cities, with all the attendant social 
problems? 

Response: As the political scientist 
Margaret Weir has pointed out, in 
American cities local land use and hous- 
ing policies, and federal and state urban 
policies, have made it easier to separate 
groups by income, race and ethnicity. 
U.S. cities have much greater autonomy 
and control over population movements 
and urban development than European 
urban areas — such as through educa- 
tional policy, zoning ordinances, and 
other strategies manifestly designed to 
ensure community life quality. Federal 
policies have supported such autono- 
my. 

For example, in the U.S. federal fi- 
nancing for freeways and subsidies for 
home ownership contributed to the de- 
velopment of urban suburbs and new 
political jurisdictions. Federal acquies- 
cence to community organizations’ re- 
sistance to public housing in middle 
class neighborhoods contributed as 
well, as did federal toleration of exten- 
sive segregation of blacks and other mi- 
norities in urban housing markets. This 
led to large-scale segregated housing 
projects that became disadvantaged mi- 
nority areas concentrated in overcrowd- 
ed, deteriorating inner-city ghettos. 

And since the 1980s, the federal gov- 
ernment has withdrawn support for ba- 
sic urban programs, which has in- 
creased the concentration of poverty in 
these urban areas. This sharp drop in 
support for such programs in turn con- 
tributed to the declining political influ- 
ence of cities and the rising influence 
of suburban political power bases. 
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In Britain, a strong central govern- 
ment has exerted significant control 
over population movements and urban 
development. Local authorities have lit- 
tle autonomy. Since the condition and 
quality of urban educational institutions 
are determined less by local govern- 
ment budgets than by national financ- 
ing, there is less competition between 
cities for scarce resources. Furthermore, 
the construction of urban housing in 
Britain has involved mainly public hous- 
ing, which made the mixing of people 
with different incomes inevitable. And 
public housing was less concentrated 
in particular areas-it was not deliber- 
ately segregated. For example, by the 
late 1980s, while 43% of public hous- 
ing was in poorer inner London, 29% 
was in central London, and 23% was 
outside London city limits. 

In France, the disadvantaged tend to 
be concentrated in city suburbs, with 
in-city housing viewed as the most de- 
sirable. Although poor ethnic and racial 
minorities tend to be concentrated in 
urban high-rise public housing, French 
land use and housing policies have con- 
trolled population movements more 
than in the U.S., and permitted more 
income and ethnic mixing than in the 
U.S. or Britain. 

This is changing, however. Recent 
population movements and government 
action in Britain and France have in- 
creased the concentration of poverty 
and racial/ethnic segregation, much as 
such policies separated populations in 
the U.S. It is unlikely that Britain and 
France will experience the degree of in- 
come and racial/ethnic separation and 
segregation that characterizes U.S. met- 
ropolitan areas, however. Central gov- 
ernments are likely to maintain control 
over the development of new political 
jurisdictions, reducing incentives and 
opportunities for such separation. Also, 
suburbanization in Europe has not tra- 
ditionally involved an abandonment of 
cities as desirable residential areas. 
Rather they are treated as national re- 
sources to be protected. 

Question: How have European anti- 
poverty policies differed from those in 
the U,S.? 

Response: It is clear that poverty and 
inequality have increased in Europe as 
well as in the U.S., related to econom- 
ic, social and political forces. However, 
by the second half of the 1980s, the 
American poverty rate among the non- 
elderly population was more than twice 







the rate of most European countries and 
almost three times the West German 
rate. The U.S., in fact , had the highest 
national poverty rate of all thirteen 
countries included in the database of 
the 1991 Luxembourg Income Study. The 
severity of poverty in the U.S., and the 
marked difference between whites and 
racial minorities in the concentration of 
poverty, were significant findings. 

The still-dramatic differences be- 
tween U.S. and European poverty are 
likely associated with varying antipover- 
ty policies. European policies have been 
directed more broadly toward the work- 
ing class. The prevailing view in Europe 
following World War II has been that 
the best policies for assisting the disad- 
vantaged are policies applied through 
programs benefiting the population as a 
whole — that is, universalistic policies, 
rather than policies leading to specifi- 
cally-targeted, means-tested programs. 
These universalistic policies have sup- 
ported macroeconomic strategies that 
address general concerns about labor 
force and labor market issues. And 
these policies have tended to be linked 
more closely with social security, health 
and housing policies reinforcing the 
working and middle classes. 

Although the influence of trade 
unionism in assisting vulnerable groups 
in the labor market has eroded both in 
the U.S. and Europe, this development 
in the U.S. reflected greater pessimism 
about alleviating poverty and mounting 
effective welfare programs. This pessi- 
mism flowed from the use of a limited 
range of antipoverty strategies due to 
American resistance to universalistic so- 
cial service and labor market interven- 
tions. Largely ignored in the U.S. were 
the effects of basic economic and tech- 
nological changes that were affecting 
the work opportunities, experiences 
and prospects of the poor. Absent from 
anti-poverty policy was the European 
concept of comprehensive programs de- 
signed to promote the social rights of 
citizenship. 

Europe’s postwar commitment to 
universal programs and to social inclu- 
sion, however, was based on assump- 
tions of steady growth and full employ- 
ment, on improvements in employment 
and labor market realities, and on no- 
tions of collective solidarity and cultural 
homogeneity, which de-emphasized 
historic divisions between religious and 
ethnic groups. These assumptions were 
weakened by economic change and the 
absorption of a larger number of immi- 
grants, making issues of race, ethnicity 



and cultural diversity more critical to 
consider. Now the universalistic Euro- 
pean welfare systems are undergoing 
significant change, with an increasing 
marginalization of certain groups and 
the potential emergence of a racialized 
economic underclass. Rising unemploy- 
ment and economic fragmentation in 
European countries pose a test of the 
ability of these welfare states to 
counter marginalizing tendencies. 

For example, an interdisciplinary 
study of poverty and marginalization in 
West Germany revealed the effects of 
economic dislocation on citizens’ social 
rights. Historically, the “productive 
core” of German society, in terms of 
stable employment, was in control of 
the social insurance system. Those 
more vulnerable to unemployment were 
in large part excluded from many facets 
of this system. Benefits for this growing 
vulnerable group were accessible main- 
ly through means-tested public assis- 
tance. Although 25% of workers were 
eligible for these programs in the mid- 
1980s, more than half of these workers 
had not applied due to the stigma at- 
tached to non-universalistic welfare 
strategies. This new division of the Ger- 
man welfare state into two types of 
workers with different welfare benefits 
has raised the possibility of ghettoizing 
marginal workers, particularly a grow- 
ing number of East Germans and an in- 
creasingly large foreign subpopulation. 
During 1991, nearly 15% of all foreign- 
ers were receiving public assistance at 
some point; less than 5% of the Ger- 
man population was doing so. In the 
context of the higher use of public as- 
sistance by foreigners and their fami- 
lies, the Solidarity Pact of 1993 involved 
cuts in means-tested programs, which 
further isolated certain groups from 
mainstream citizen rights and benefits. 

In the United Kingdom, the Thatch- 
er government consciously exacerbated 
the division of the welfare state by en- 
couraging tax-supported private and 
company-based welfare among the 
middle class and more affluent work- 
ers, along with major cost cutting in so- 
cial security and housing. This shifted 
the welfare state further toward more 
targeted welfare programs. The same 
trend has occurred in France and the 
Netherlands. Nevertheless, as I pointed 
out earlier, although there is a trend to- 
ward the more limited social provision 
of the U.S. system, the European com- 
mitment to universal social services and 
broad family policies remains a firm in- 
stitutional feature. 
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Question: What do you mean in the 
book by the emergence of “ new poverty ” 
in the U.S. and Europe? 

Response: The period since the late 
1970s marks an important juncture in 
addressing poverty and inequality is- 
sues in the western world. Economic 
insecurity has been accompanied by 
family disintegration and altered forms 
of public assistance, reducing the life 
chances of an increasingly vulnerable 
minority population. A variety of eco- 
nomic, social and political forces have 
been producing powerful new configu- 
rations of inequality which we have re- 
ferred to as "the new poverty.” These 
configurations reflect changes in the 
size and composition of economically 
marginal groups, the crystallization of 
racial cleavages among them, a down- 
ward turn in their life chances, and an 
increase in their social and political iso- 
lation. 

As poverty rates have risen in most 
western societies, the rates among 
young people and prime-age workforces 
have grown disproportionately, particu- 
larly among single parents with chil- 
dren. In the U.S., the movement away 
from manufacturing toward a service 
economy, the increase in the proportion 
of low-wage and part-time employ- 
ment, widespread worker dislocation, 
and rising long-term unemployment in 
certain segments of the labor force 
have been particularly traumatic for the 
urban black poor because of their con- 
centration in deteriorating inner-city 
ghettos. Consequently poverty is affect- 
ing the opportunities and future of the 
poor more dramatically than in the re- 
cent past. Some experts see a “femini- 
zation” and “racialization” of poverty 
which revolves around labor market 
and labor force relationships. 

In this new environment, the “moral 
worth” of the poor is scrutinized more 
closely in means-tested programs — 
that is, their work ethic, family relation- 
ships and integrity. This is changing the 
poor’s attitudes toward government 
and the welfare system, with these in- 
stitutions being viewed as hostile adver- 
saries. Rather than the welfare system 
having evolved into a "dependency cul- 
ture,” research indicates that it is be- 
coming a "captivity culture” in which 
welfare claimants may be trapped into 
dependency against their will and poli- 
cies designed to promote self-sufficien- 
cy may reinforce dependency as a man- 
ageable phenomenon. 



Another critical factor in under- 
standing the new poverty is the mas- 
sive breakdown of social institutions in 
poverty neighborhoods. An important 
aspect is the absence of nonpoor fami- 
lies in ghetto areas which could serve 
as conventional models in socializing 
poor youth. The absence of diverse so- 
cial resources and crucial community 
networks has weakened youth’s attach- 
ment to the legitimate labor market. 

Question: What do you see as the ma- 
jor characteristics of American anti- 
poverty policies and how effective do 
you feel they have been? 

Response: In 1975, 30% of all Ameri- 
can poor had incomes below 50% of 
the official poverty level; in 1988, 40% 
had such incomes. The increase was 
much greater for blacks — in 1975, 
nearly half had incomes at that level. 
These effects are strongly tied to post- 
war American antipoverty policies that 
have led to relatively autonomous ar- 
rangements for the poor in isolation 
from broader social concerns about eco- 
nomic, educational and employment is- 
sues. In fact, we have focused on de- 
veloping programs rather than on 
developing national policy — particular- 
ly decentralized and non-integrated pro- 
grams and demonstration projects. 

Historical analyses of the American 
approach to poverty consider this ap- 
proach a legacy of the New Deal. The 
nationwide social security system was 
an extension of the rights of a predomi- 
nant group of citizens, the “deserving” 
working class. They had the support of 
the middle class and a strong federal 
bureaucracy. However, the New Deal 
did little for those with low skills and 
social status, including blacks moving 
to northern cities from the rural south. 
Attempts to develop a national full em- 
ployment policy and health care pro- 
gram in the 1930s and 1940s, which 
could have assisted the disadvantaged, 
were not politically viable. Neither 
were U.S. laws, union power or cultur- 
al traditions supportive of the disadvan- 
taged, as they were in Europe. Assis- 
tance to this group was confined to 
poorly coordinated second-tier programs 
under the "welfare” system. This re- 
stricted the scope of anti-poverty strate- 
gies and reinforced the American divi- 
sion between the "undeserving” and 
"deserving” poor, the former acquiring 
the status of a class apart. 

In many ways, the Great Society 
programs of the 1960s continued this 



division. Although government benefits 
were improved, little was proposed for 
integrating clients into the social and 
economic mainstream. Since poverty 
was not viewed as a national economic 
problem, antipoverty programs focused 
on efforts to change the behavior of the 
poor rather than their economic oppor- 
tunities. 

The targeted programs that became 
the traditional American response to 
poverty created a policy dualism that 
made it easier for the middle and upper 
classes to criticize the welfare system 
and its clients. The welfare backlash of 
the 1980s and 1990s is also a product of 
antipoverty strategies that fostered in- 
tergroup misunderstanding and mis- 
trust, as well as the isolation of the 
poor. As one expert put it, we are in a 
“retreat from caring.” To many middle 
Americans, the poor had become "the 
welfare class” — that is, the bottom 
level of the stratification system defined 
in terms of their behavioral and cultural 
deficiencies. This "class” was consid- 
ered to pose a growing threat to public 
norms and safety. The emphasis on the 
personal characteristics of individuals, 
particularly their work ethic, ignored 
the large-scale economic changes that 
were contributing to increasing poverty. 
At the same time, research was strong- 
ly suggesting that there was no conclu- 
sive evidence for prevailing beliefs that 
welfare participation discourages work, 
divides families or increases the child- 
bearing rates of unmarried women. 

Two themes have now emerged: 

1) welfare receipt should involve cer- 
tain responsibilities on the part of the 
recipient, and 2) able-bodied adults re- 
ceiving welfare should be required to 
develop educational and work skills, to 
search for jobs, and to accept the jobs 
offered them. Both themes are based 
on pre-existing assumptions about the 
moral fabric of individuals, independent 
of the social, economic, cultural and 
political structure of society. 

While economic restructuring and 
downsizing have had adverse effects 
on lower-income groups in all advanced 
industrial nations, the effects of jobless- 
ness on the American poor have been 
far more severe than elsewhere be- 
cause of the underdevelopment of the 
welfare system and the weak institu- 
tional structure ensuring the general so- 
cial rights of citizens. The tradition of 
noncomprehensive programs in the U.S. 
is particularly problematic for poor in- 
ner city blacks who have been handi- 
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capped by intergenerational discrimina- 
tion and find themselves isolated in 
ghetto neighborhoods. 

Question: What conclusions do you 
come to in the book about effective anti- 
poverty policies and strategies? 

Response: Any program proposing to 
improve the life chances of the poor, 
including the ghetto poor, needs to be 
based on concerns about both the poor 
and the working classes. Unemploy- 
ment, job security, declining wages, ris- 
ing medical and housing costs, the 
need for child care, the sharp decline 
in the quality of public education, 
crime and drug abuse — these broader 
problems must be addressed. Reliable 
national surveys indicate wide support 
for general strategies to resolve employ- 
ment problems, such as educational 
and skill training, the provision of day 
care, the reform of the health care sys- 
tem, and crime and drug addiction pre- 
vention. This suggests possibilities for 
new demographic and political align- 
ments and coalitions that can place 
greater emphasis on the integration of 
programs promoting social and econom- 
ic progress for all groups in society, not 
just disadvantaged segments. 

In Europe, income transfers, family 
policies and health benefit systems re- 
ceive wide popular support, and in- 
clude an array of related program strat- 
egies. In the Swedish reform package, 
labor market programs still play a key 
role. In Germany, the tradition of active 
labor market programs and close linkag- 
es across government, educational insti- 
tutions and industries have continued, 
but with significant cost cutting. When 
Germany’s vocational vs. higher educa- 
tion dual system was failing to move 
many young people into permanent 
mainstream employment, a well-direct- 
ed trade union and employer effort has 
begun to restore its health. In France, 
the United Kingdom and the Nether- 
lands, there has been a movement to 
improve job skills and opportunities 
even though there has been a negative 
reaction to the welfare state in terms of 
its perceived role in increasing depen- 
dence. 

Unfortunately many of the reforms 
that focus on more universalistic strate- 
gies, such as labor market programs, 
have been poorly designed and their 
delivery suffers from unprecedented de- 
centralization. These programs can be 
effective only as part of broad national 
programs promoting economic recovery 
and adjustment to structural change . 



Editor’s Note 



The first issue of Evaluation Forum 
focusing on poverty featured an inter- 
view with William Julius Wilson based 
on his classic, The Truly Disadvan- 
taged For those interested in a reprint 
of that interview, please contact the ed- 
itor of Evaluation Forum . 

Readers may also be interested in 
the following books and articles au- 
thored, co-authored or co-edited by 
Wilson: 

1987: The Truly Disadvantaged: The Inner 
City The Underclass, and Public Policy 
Chicago, IL: University of Chicago 
Press. 

1991: "Studying Inner-City Social Disloca- 
tions: The Challenge of Public Agenda 
Research,” in American Sociological Re- 
view 56. 

1 995: Poverty, Inequality, and the Future of 
Social Policy: Western States in the New 
World Order . (W.J. Wilson with K. Mc- 
Fate and R. Lawson, editors.) New York, 
NY: Russell Sage Foundation. 

1996: When Work Disappears: The World of 
the New Urban Poor. New York, Alfred A. 
Knopf. 
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Additional books and articles on 
comparative poverty issues: 

Brown, J. (editor) Anti- Poverty Policy in the 
European Community London, UK: Policy 
Studies Institute, 1984. 

Glazer, N. and K. Young (editors) Ethnic Plu- 
ralism and Public Policy Lexington, MA: 
D.C. Heath, 1983. 

Katz, M. The Undeserving Poor: From the 
War on Poverty to the War on Welfare . 
New York, NY: Pantheon, 1989. 

Klass, G. “Explaining America and the Wel- 
fare State: An Alternative Theory,” in Brit- 
ish Journal of Political Science 15, 1983. 

Korpi, W. “ Social Policy and Distributional 
Conflict in the Capitalist Democracies,” 
in West European Politics 3, 1980. 

OECD: The Future of Social Protection: The 
General Debate. Paris, France: OECD, 
1988. 

Rainwater, L. Poverty in American Eyes. 
Luxembourg Income Study, 1991. 

Standing, G. Unemployment and Labour 
Market Flexibility: The United Kingdom. 
International Labour Office. Geneva, 
Switzerland, 1986. 

Weir, M. “Race and Urban Poverty: Com- 
paring Europe and America.” Occasional 
paper. Center for American Political 
Studies. Cambridge, MA: Harvard Uni- 
versity Press, 1993. 

Weir, M., S. Orloff, and T. Skocpol (editors) 
The Politics of Social Policy in the US. 
Princeton, NJ: Princeton University 
Press, 1988. 
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Editor’s Introduction 



In March, 1996, the U.S. General Ac- 
counting Office (GAO) published a re- 
port on at-risk and delinquent youth. It 
was revealing, if not shocking. Accord- 
ing to the GAO, approximately 20% of 
children in the U.S. live in poverty; for 
minority children the percentage is 
twice as high. Poor teens are more than 
three times as likely as other adoles- 
cents to drop out of school. While the 
high school dropout rate has been fall- 
ing, in 1993 it remained over 10% for all 
races combined, and over 27% for His- 
panic Americans. 

The unemployment rate for youth is 
much higher than for adults. From 1991 
to 1994, the average unemployment 
rate among males 16 to 24 years old 
was 14.3%, compared with 5.7% for 
those aged 25 and older. During the 
same period, the average unemploy- 
ment rate among females 16 to 24 was 
12.3%, compared with 5.3% for those 
25 and older. 

In 1992, child protective service 
agencies received approximately 1.9 
million reports of child abuse and ne- 
glect. In 41% of these reports, investi- 
gations confirmed maltreatment. Re- 
search has suggested that such abuse 
and neglect increases the likelihood of 
juvenile arrests by 53%. Between 1988 
and 1992, the juvenile arrest rate for vi- 
olent crime increased 47% — more 
than tvyice the increase for people 18 
years of age and older. The number of 
violent crimes committed against youth 
aged 12 to 17 increased 25%. In 1991, 
youth aged 12 to 23 were more likely to 
be victims of a violent crime than were 



people past their mid-20s. In 1992, 
roughly one out of every thirteen juve- 
niles reported having been a victim of 
a violent crime. Meanwhile, the U.S. 
has the highest teen alcohol-related 
and drug abuse-related rate of any in- 
dustrialized nation. 

The federal government currently 
operates 131 programs intended to 
benefit youth, including at-risk and de- 
linquent youth. These programs are ad- 
ministered by 16 different federal de- 
partments and agencies. Many of these 
programs have similar goals, and pro- 
vide multiple services to multiple target 
groups, although the majority target all 
youth.The GAO indicates that the feder- 
al government spent over $4 billion on 
programs for at-risk and delinquent 
youth in 1995. 

The Congressional request to the 
GAO was simply to list and describe all 
these federal programs without offering 
recommendations. The clear intent of 
the House Committee on Governmen- 
tal Affairs in making the request, how- 
ever, was to determine subsequently 
what programs could be eliminated or 
consolidated. It is interesting, in this 
context, that the GAO chose to provide 
extremely troubling data on the status 
of youth in America in the mid-1990s. 

The section that follows provides 
some international perspectives on 
youth issues, which help place the 
American statistics within a broader 
context. It also addresses strategic 
planning and methodological issues in 
analyzing the “performance measure- 
ment movement” and its relationship to 
evaluation. 







An Overview of 
Research on 
International Youth 
Issues 



Two books on international youth is- 
sues provide interesting ideas to consid- 
er in terms of youth development and 
programming. One was published by 
the Organization for Economic Coopera- 
tion and Development (OECD), the oth- 
er by the United Nations. 

The United Nations 

The International Year of Youth was 
celebrated in 1985. This nominal focus 
on the world’s youth prompted the 
United Nations to initiate the United 
Nations University’s Project on Youth . 
The project produced a book reviewing 
expert opinion and research on youth 
issues titled Perspectives on Contempo- 
rary Youth. Edited in 1988 by J. Kuczyn- 
ski, S. Eisenstadt, B. Ly and L. Sarkar, it 
featured a chapter by E. Allardt which 
may be of special interest to readers, 
“The Current Context of Youth in Econ- 
omy, Politics and Societal Develop- 
ment.” 

The Transition from Youth to 
Adulthood 

Allardt maintains that although the 
length and nature of the transition peri- 
od from childhood to adulthood has 
varied in different societies, it is today 
associated with more serious prob- 
lems — “being young and becoming an 
adult seem to be more difficult than 
during many other periods in history.” 
This transition is highly dependent, he 
says, on economic and social condi- 
tions. He reminds us that there are sig- 
nificant differences in opportunity for 
youth in rich vs. poor countries. For ex- 
ample, although unemployment is a 
problem everywhere, in developing 
countries underemployment or “ill-em- 
ployment” is the dominant characteris- 
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tic. Currently, even in the industrialized 
countries, unemployment has reached 
unexpected high levels, particularly 
among youth. 

Technical improvements in agricul- 
ture are leading to greater rural unem- 
ployment and rising migration to urban 
centers. Employment has shifted from 
agriculture to manufacturing and now 
to services. Approximately two-thirds of 
the U.S. labor force is currently in the 
service sector; approximately 60% is in 
services in Western Europe. Electronic 
data processing is used increasingly in 
all sectors of economies. 

These major changes, Allardt says, 
have had a number of consequences 
for the transition of youth to adulthood. 
Changes in the structure of employ- 
ment which have restricted opportuni- 
ties for social mobility, have affected 
youth’s view of their career opportuni- 
ties and life chances. There has been a 
substantial expansion of government, 
with the public sector assuming many 
of the social functions formerly per- 
formed by family and community. Con- 
sequently current attempts to reduce 
the size and change the nature of pub- 
lic social safety nets will likely have a 
negative effect on the employment pos- 
sibilities of young people. 

Allardt proposes that economic 
change, change in the structure of em- 
ployment, and an increased role for 
government has influenced youth atti- 
tudes. Some of the elements of youth’s 
new orientation are these: 

■ An emphasis on inner-direction, 
compared with sensitivity to manip- 
ulation by external demands. 

■ A stress on individual human rights. 

■ A focus on protecting the fate of the 
individual -- i.e. individual safety, 
physical, psychological and social. 

■ An interest in causes, without con- 
sideration of material interests, well- 
being or rights. 

These new attitudes, particularly 
among youth in industrialized coun- 
tries, have been played out in anti-bu- 
reaucratic, environmental, anti-nuclear, 
feminist and peace movements, as well 
as in religious and ethnic revivals, ac- 
cording to the author. These attitudes, 
he says, appear to be reactions against 
structural and institutional trends: the 
increasing importance of service and 
goods; the penetration of government 
into most spheres of life; and the in- 
creased size of the well-educated mid- 
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die class in the context of the decline of 
its socioeconomic position. 

The Youth Labor Market 

Allardt sees the nature and degree of 
youth unemployment as conditioned 
strongly by demographic factors. Many 
developing countries have a larger 20-24 
age group than can be absorbed by their 
labor markets. In advanced industrial- 
ized countries, the “baby boom” gener- 
ation entered the labor force in the mid- 
1970s when labor demand was weak. 
Evidence from a number of countries 
reveals a growing trend toward long- 
term unemployment in the 20-24 age 
group. This shift in the age structure of 
unemployment, Allard believes, is likely 
to continue. 

In fact, research indicates that youth 
unemployment may be more troubling 
than the statistics suggest. Youth have a 
tendency to leave the labor market 
when facing difficulties. In most coun- 
tries there is a sizable minority of young 
people who experience serious prob- 
lems during the transition from youth to 
adulthood and consequently live a job- 
lessness or casual labor employment ex- 
istence. Those with low educational at- 
tainment and physical and emotional 
handicaps, and ethnic minorities and 
teenage parents, are overrepresented in 
this group. 

Allardt points out that maintaining 
young people in educational institutions 
has been one of society’s main devices 
for coping with the problem of unem- 
ployment. The amount of formal educa- 
tion now required by postindustrial la- 
bor markets assigns additional tasks to 
these institutions — intellectual, aes- 
thetic, ethical, psychological, and other 
tasks. At the same time, the motivation 
to learn is becoming increasingly depen- 
dent, he says, on the characteristics of 
education (its quality) rather than on its 
prospective rewards. 

A recent study by OECD drew the 
conclusion that youth’s most powerful 
demand was for full integration into the 
adult world, and that this demand has 
been moving down the age scale. In- 
creasingly younger age groups have 
now ceased to define themselves as 
children, while the upper limits of sec- 
ondary and postsecondary education 
have moved upwards — consequently 
the demand for adult status and the real 
possibility of acquiring it are farther 
apart. Even adults can no longer feel 
fully prepared for rapid economic and 



labor market change in terms of educa- 
tion and skills. Therefore the bound- 
aries between adults and youth have 
grown vaguer. 

Although Allardt takes the position 
that young workers are more sensitive 
to working conditions than their elders 
— both physical safety and alienation — 
he also points out that youth are in the 
forefront of technological change, which 
translates to new job opportunities. 

What the trade-off will be between such 
change and unemployment is yet to be 
determined. 

The Organization for Economic 
Cooperation and Development 

The status of youth in European 
Union countries and other industrialized 
nations has been a longstanding inter- 
est of the OECD. In 1989, representa- 
tives of seventeen OECD member coun- 
tries met in Paris to develop a project 
on at risk youth under the sponsorship 
of OECD’s Centre for Educational Re- 
search and Innovation (CERI). This 
meeting was followed by two interna- 
tional seminars supported by the re- 
search arm of the U.S. Department of 
Education and several major founda- 
tions. 

The project focused on three educa- 
tional/training periods: preschool , 
school , and the transition to work. Its 
purpose was to identify key issues in 
studying youth problems, and to make 
recommendations for national and inter- 
national youth policy. In 1995, CERI 
produced a book based on the study 
commissioned by OECD, Our Children 
At Risk. The terms “at risk” referred to 
youth from disadvantaged backgrounds 
who are likely to fail to meet school 
standards, often drop out of school, 
and frequently fail to become integrat- 
ed into a normally accepted pattern of 
social responsibility, particularly with 
regard to work and family life. 

School-to-Work Cooperation 
and Linkages 

The European Union has been trou- 
bled in the 1990s by the proportion of 
at risk youth in member countries. Up 
to thirty percent of school age children 
are estimated to be “at risk.” Even in 
countries where unemployment has 
fallen there has been a rise in unem- 
ployment for ethnic minorities and 
youth. Any gains made in the educa- 
tional system are sometimes sacrificed 
in the transition period from school to 
work. 
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European countries have traditionally 
relied on strong relationships between 
schools, industries and businesses in 
terms of developing relevant skills. For 
example, in the United Kingdom, pro- 
grams have been developed to give 
school curricula a more job-oriented di- 
mension along the lines of the Boston 
Compact. In France, the educational sys- 
tem emphasizes the consistent guidance 
and monitoring of youth, work study 
programs, and an individualization of 
educational plans. In Sweden, students 
are required to have a series of weeks in 
a work experience setting. In Japan, 
special schools and training colleges 
linked with businesses have been devel- 
oped to offer flexible opening times for 
students prior to and following work, 
and alternatives to drop-outs. Often 
community initiatives have supplement- 
ed the above linkages. In the United 
Kingdom, special programs for inner 
city youth have been developed. In Aus- 
tralia, community centers have been es- 
tablished. 

In the Netherlands, new policies link 
a number of government ministries 
around these purposes: 

■ To coordinate national policies on 
youth education. 

■ To collaborate with trade unions and 
employers around more effective uti- 
lization of existing programs and in- 
frastructures. 

■ To combine budgets. 

In Australia, cooperation across 
schools, universities, research organiza- 
tions, trade unions and businesses is 
part of a general strategy to broaden 
school-to-work options. In Germany, co- 
ordination policies involve welfare 
agencies, businesses, schools, and em- 
ployment and training programs. In Por- 
tugal, there is extensive development of 
multi-agency community programs to 
encourage the social and economic inte- 
gration of minorities. 

However, a lack of cooperation and 
coordination across multiple actors con- 
tinues to be a major barrier to serving 



at risk youth adequately, according to 
the OECD report. Nevertheless, some 
countries have developed useful mod- 
els. For example, Portugal has devel- 
oped an interministerial program that 
unifies the national approach at central, 
local and school levels, and is coordi- 
nated by a central interministerial coun- 
cil. This approach gives priority to basic 
education and to areas of low educa- 
tional performance, and involves a link- 
age between the educational system and 
health and nutrition programs, recre- 
ation programs, family support pro- 
grams, and transportation systems. 

Some Conclusions 

Extending the capabilities of school 
systems in responding to the needs of 
disadvantaged students has led many 
countries to review the balance of con- 
trol between central and local de- 
mands. Many countries see the mainte- 
nance of quality as a significant 
challenge. Some have resolved this is- 
sue by emphasizing local control while 
requiring some form of accountability to 
the central government. 

An array of intervention strategies 
have been developed in OECD coun- 
tries, including service coordination, in- 
creasing continuity across major transi- 
tion periods, using flexible approaches, 
giving greater autonomy to schools, 
changing school organization, develop- 
ing new curricula, assigning more im- 
portance to youth’s personal and social 
needs, and giving more freedom to 
teachers to develop more effective 
teaching strategies and better linkages 
with employers. These interventions of- 
ten involve systemic changes in the 
school-to-work sequence, and its insti- 
tutions and organizations, and the use 
of educational and training resources. 



Editor’s Note 



Please see Part II of the Resources 
Section for international references on 
youth issues. 



YouthTraining 

An interesting cross-national study 
supported by the U.S. Department of 
Labor has been conducted by The 
Rand Corporation to study the self-re- 
ported formal job training and labor 
market outcomes of male youth in the 
United States, Britain and Australia. 

The report on the study was published 
in 1991. Three national surveys provid- 
ed the data: the U.S. National Longitu- 
dinal Survey of Young Men (NLS), the 
British National Child Development 
Study (NCD) and the Australian Longi- 
tudinal Survey of Youth (ALS). Informa- 
tion was available on educational at- 
tainment, participation in formal 
training programs, wages, job charac- 
teristics and employment. In addition, 
panel data on other relevant variables 
was analyzed. The study was guided 
by propositions about the relationship 
between determinants of training and 
the effects of training on wages and 
the likelihood of employment. 

Research Questions 

Several interesting policy questions 
directed the study: 

■ Do youth compensate for a low level 
of education through job training, or 
do those lacking education lose ac- 
cess to future occupational training? 

■ What role does technology play in 
determining workplace educational 
and skill needs? 

■ Do high union wages and work rules 
reduce employers’ incentive to pro- 
vide training? 

■ How do training patterns differ in 
the three countries and what policy 
lessons are suggested by commonali- 
ties and differences? 

Study Results 

A statistical analysis of the economic 
determinants of training in the three 
countries indicated that the probability 
of receiving most kinds of formal train- 
ing rose with the level of schooling re- 
ceived, with the possible exception of 
Australia where education and training 
appeared to complement rather than 
substitute for one another. One implica- 
tion of these results was that youth with 
low schooling also confront limited 
training opportunities and consequently 
a slower growth in wages. The probabil- 
ity of obtaining training, however, 
peaked at different schooling levels for 
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each source of training, and training 
probabilities varied by apprenticeship 
status in Britain and Australia. 

There was a dramatic pattern of ef- 
fects on training by the source of train- 
ing. The likelihood of company training 
was greater in high technology jobs, es- 
pecially for those with more education. 
Rapid technological change clearly pe- 
nalized early school leavers. High tech- 
nology jobs were correlated also with a 
lower likelihood of off-the-job training. 
These results suggested that in a grow- 
ing and technologically progressive en- 
vironment, employers rely more on 
company training for skill needs than 
on schools and other outside sources of 
training. 

For American male youth, the proba- 
bility of receiving training from all 
sources rose with years of work experi- 
ence, holding job tenure constant. For 
British youth, only company-based 
training appeared to rise with work ex- 
perience, and then to decline. And the 
likelihood of training from most other 
outside sources decreased with work ex- 
perience . 

Employers appeared to use different 
training strategies in the three coun- 
tries. In the U.S., after a beginning 
training period in the first year on a 
job, training from schools and busi- 
ness/technical institutes decreased 
with time in the job, while company- 
based training and training from other 
sources rose with tenure. This pattern 
suggested that company training was 
substituted for broad-based general 
skills acquisition supplied elsewhere. 
British and Australian employers tended 
to rely on outside training sources, with 
company training falling with job ten- 
ure, while off-the-job training rose ini- 
tially and then declined. 

Unexpectedly, the study showed 
that in all three countries union mem- 
bership or coverage was associated 
with a greater probability of receiving 
formal training from most sources. This 
suggested a more extensive training role 
for unions than had been anticipated. 

For all countries, the probability of 
receiving training from most sources 
was usually reduced for 1) those who 
had gained work experience prior to 
the completion of schooling, and 
2) those with more than three previous 
jobs or those having recently changed 
jobs. The researchers concluded this 
may have been because youth were 



bringing skills from previous jobs to the 
new one. 

Controlling for other factors, white- 
nonwhite status did not appear to affect 
access to most formal training. High un- 
employment increased the likelihood of 
being trained in a business or technical 
institution in the U.S., but reduced 
training for nonapprentices in Britain. 
Respondents in large firms were more 
likely to receive formal training from all 
training sources. Private sector employ- 
ment in both Britain and Australia, 
however, was usually associated with a 
lower likelihood of receiving training. 

The research showed that in the U.S. 
wages rose with both work experience 
and job tenure, although at a slower 
rate at the higher work experience lev- 
els. Results for Britain suggested that 
each year of work experience increased 
wages by about the same amount as in 
the U.S. In Australia, wages grew rapid- 
ly with years of work experience but the 
effect lessened quickly. Job tenure had 
no statistically significant effect on 
weekly wages in Britain or Australia. 
Much of the cross-national difference in 
wage profiles seemed to be due to the 
larger wage-tenure effects in the U.S. , 
which may mean there is considerably 
more informal on-the-job training in 
the U.S. than in the other two countries. 

Training appeared to produce higher 
marginal wage returns to labor market 
entrants with low schooling, possibly 
because training tended to substitute 
for school at that level of educational 
attainment. Also, a large part of the 
training provided seemed to be quite 
general. Transferable skills appeared to 
be related more to time in the labor 
market than to job tenure. The wage 
returns for a high level of schooling 
were clearly higher in high-technology 
jobs. Technological change actually re- 
duced the wages of the least educated. 
The research suggested that better-edu- 
cated workers, in all three countries, 
were better able to respond to techno- 
logical change, and were more produc- 
tive and more highly rewarded. 

In each country, wages for nonwhite 
male youth were 10% lower than for 
other groups. Union members generally 
received higher wages. Large firms paid 
more. Training had no apparent effect 
on the growth of wages, but each pre- 
vious training episode tended to raise 
subsequent wages. Company-based 
training had the largest effect on wages 
in the U.S., followed by training from 
business/technical institutions. For the 



U.S., training courses taken in institu- 
tions following the completion of 
schooling had no apparent effect on 
wages. The results were similar for the 
other two countries. However, the size 
of wage returns associated with the re- 
ceipt of training revealed large differ- 
ences across the three countries. The 
wage effects of formal training in the 
U.S. was about twice those in Britain 
and Australia. This could explain why 
employers and young workers estab- 
lished in the workforce in these coun- 
tries have less incentive to provide and 
receive training, since the returns for 
schooling and other forms of invest- 
ment may be as high as those for for- 
mal training. 

In all three nations, the likelihood of 
experiencing a spell of unemployment 
in a given year was lower for those 
with higher schooling, more general 
work experience and longer job tenure. 
Employment in large firms was associ- 
ated with greater stability. Membership 
in unions was related to a higher proba- 
bility of unemployment. Higher rates of 
technological change in the U.S. and 
Britain were associated with a lower 
likelihood of unemployment. In the 
U.S. this was true mainly for selected 
schooling groups, suggesting that there 
was less technology-induced youth un- 
employment than expected. 

Training reduced the likelihood of 
unemployment in all three countries. 
Each training episode appeared to low- 
er the probability of future unemploy- 
ment, particularly if it was job-related, 
with a slight decline in this effect over 
time. 

Conclusions 

In all three countries, the probability 
of receiving training from all sources in- 
creased with the level of educational 
attainment. Union involvement in- 
creased access to training. Although 
American male youth received much 
less training at entry to the workforce, 
they accumulated training at a much 
more rapid pace with time in the labor 
force. In fact, their likelihood of receiv- 
ing additional company training re- 
mained high, in contrast with the de- 
clining access of British and Australian 
youth to company training as work ex- 
perience increased. 

Although the analyses were confined 
to the study of formal training reported 
by survey respondents, and excluded 
potentially important investments in in- 
formal forms of training and learning on 






Evaluation Forum ■ Issue 12 ■ Summer 1997 



Evaluation Issues and Activities ■ International Perspectives on Youth Issues 



the job, significant insights were ac- 
quired: 

■ American youth receive less formal 
training on entry to the labor market 
but more formal training with time 
in the labor market. 

■ In the U.S., employers provide 
young workers with training that is 
increasingly company-based, relying 
less on outside sources as workers 
acquire job tenure — which responds 
well to technological change. 

■ The level of school attainment is an 
important predictor of post-school 
training and labor market success. 
Better-educated youth are consider- 
ably more likely to receive training, 
especially company training, which 
has the greatest influence in raising 
wages and reducing the likelihood of 
unemployment. 

■ Technological change plays a critical 
role in raising educational and skill 
requirements. 

■ Technological change increases the 
likelihood of receiving company 
training, particularly for well-educat- 
ed youth, and reduces the probabili- 



ty of training from schools and other 
off-site sources. 

■ Less-educated youth receive little 
formal post-school training and are 
less likely to receive training in in- 
dustries experiencing rapid techno- 
logical change. 

■ Union membership is associated 
with an increased probability of for- 
mal training; however, this may in- 
hibit the receipt of informal training. 

■ Company-based training had the 
largest positive effect on youth wag- 
es and in reducing youth unemploy- 
ment. 

■ The positive wage effects for formal 
training in the U.S. were twice those 
for Britain and Australia. 

Some Implications 

Given the apparent strong link be- 
tween education and training, the high 
dropout rates for American youth, par- 
ticularly minority youth, pose a serious 
problem for policymakers. The gap be- 
tween the educated and less educated, 
and between whites and minorities, 
may be exacerbated without a resolu- 



tion of this problem. Meanwhile, tech- 
nological change is likely to increase 
this disparity. The study’s conclusions 
call for preventive policies and remedial 
programs to reduce dropout rates and 
increase academic proficiency in K-12 
education. 

A more specific issue raised by the 
study is the comparative feasibility and 
effectiveness of company-based train- 
ing and programs such as JTPA that 
rely mainly on off-site training. Equity 
as well as efficiency is at stake; the 
economically disadvantaged have less 
access to company-based training. 

Also, since company-based training is 
sometimes firm-specific and nontrans- 
ferable, the question of whether gov- 
ernment should subsidize company- 
based training is an additional issue of 
importance. 



Editor’s Note 



For those who want more detail, 
please see: 

Tan, H. B. Chapman, C. Peterson, and A. 
Booth. Youth Training in the U.S., Britain, 
and Australia. Santa Monica, CA: The 
Rand Corporation, 1991. 
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The Performance Measurement Movement 



Editor’s Introduction 



There is nothing new or mystical 
about the term “performance” It has a 
familiar and ancient aroma. Adam 
Smith surely talked about it. At base, it 
is a private sector concept. We talked 
about the performance of the Model A 
Ford several decades ago, and we want 
to know about the performance of the 
Toyota Celica in the 1990s. We talked 
about the performance of certain ma- 
chine parts in assembly lines during 
the industrial revolution, and we now 
need to determine the performance of 
the computers of this decade. 

What we have not tended to talk 
about is the performance of social in- 
terventions. These initiatives that flow 
from social policies have been expect- 
ed to be successful or unsuccessful, or 
in more sophisticated terms their out- 
comes have been judged efficient or ef- 
fective, but we have not talked about 
the performance of programs until quite 
recently. We not only transferred this 
word from the linguistics of the private 
sector but indulged in transnational bor- 
rowing — from Europe, Australia and 
other industrialized countries. The word 
came into its own in two worlds in the 
1980s, the private and the public, and 
in the 1990s there is a veritable perfor- 
mance-oriented movement underway in 
postindustrial governments. 

A number of nouns find themselves 
commonly used in discussions of public 
performance — performance measure- 
ment, performance indicators, perfor- 
mance targets, performance standards, 
performance management systems, 
performance measurement systems, 
performance standards systems, and 
so on. What does it all mean, and what 
is the benefit of focusing so much time 
and energy on performance? 

The movement toward the measure- 
ment of program outcomes is at the 
heart of what we will call the perfor- 
mance measurement movement. In an 
environment of high government defi- 
cits and debts and pressures to down- 
size and restructure both the private 
and public sectors to maintain societies 



in a globalized economy, accountability 
for tax money spent has become an in- 
creasingly desired commodity at all lev- 
els of government. Accountability in the 
1 990s is now defined as high perfor- 
mance. In the private sector it would in- 
volve high productivity and the sale of 
products or services at a high level of 
market share. In the public sector it in- 
volves the achievement of proxies for 
social program goals. 

Just as the private sector must mon- 
itor and evaluate products and markets, 
government must now monitor the out- 
comes and longer-term effects of social 
programs. Until the late 1970s, govern- 
ment focused more on “processes” than 
on the “products” resulting from inter- 
vening in the lives of individuals, 
groups and communities in an effort to 
enhance life quality and stabilize soci- 
ety. The term “performance” is the cho- 
sen label for this new emphasis. But 
strip the name away, and you have 
“program outcomes” and “program im- 
pacts,” old familiar concepts. 

What is novel is greater attention to 
measurement issues in developing out- 
come and impact indicators. That 
seems solidly beneficial. What is also 
new is the development of targets , 
standards and their associated rein- 
forcement mechanisms, incentive/sanc- 
tion systems. The definition of value 
here is more controversial. These 
trends represent a renewal of interest 
in what strategic planning can and 
should mean and assure. That is com- 
mendable. But the trends do not neces- 
sarily support the further elaboration 
and use of evaluation research in 
studying processes or program effects. 
That is misguided and unfortunate. 

In this series of articles, we move 
beyond the reviews concerning perfor- 
mance management in Issue #1 1 of 
Evaluation Forum , but hopefully recall- 
ing the elements of this managerial re- 
form detailed in that issue. The key 
words in those reviews were strategic 
planning, performance measurement, 
quality assurance, customer satisfac- 
tion, and continuous program improve- 
ment. You will see most of them again. 
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The articles in this series lack the in- 
tegration readers have grown accus- 
tomed to in Evaluation Forum, but there 
is a modicum of logic in their selection. 
They address different aspects of a 
larger strategic planning process, of 
which performance measurement is a 
key dimension. In this sense, total qual- 
ity management, short- and longer- 
term goal setting, the application of 
goal achievement strategies, the evalu- 
ation of the implementation of these 
strategies and of their effects, and con- 
tinuous quality improvement are interre- 
lated components of comprehensive 
strategic planning. The development of 
performance goals, targets and stan- 
dards to monitor the progress of this 
planning process and enhance it, and 
the evaluation of the efficiency and ef- 
fectiveness of the strategy in meeting 
goals, cannot be — should not be — 
analyzed in isolation from the other 
components. 

Consequently, in the first article in 
this series we look at the definitions of, 
and the relationship between, evalua- 
tion and continuous quality improve- 
ment. The second article uses a case 
example — educational skills stan- 
dards — in developing a standards- 
driven performance assessment sys- 
tem for human service initiatives and 
programs. The level and kind of skills 
attainment are important outcomes to 
be measured in judging the perfor- 
mance of education and training pro- 
grams, and in studying the longer-term 
employment outcomes of employment- 
relevant initiatives. 

The third article focuses on the fed- 
eral government’s large-scale effort in 
the 1990s to move the assessment of 
government activities away from an ex- 
clusive interest in process issues, and 
toward a judgment of performance (out- 
comes). This little-publicized initiative 
has created an assessment model ap- 
plicable across the federal agencies, 
with significance for strategic planning 
at the state and local levels. It proposes 
a performance measurement system 
with broad utility, without ignoring the 
problem of implementing such a system 
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in the real world of politics and bureau- 
cracies. 

The fourth article directs our atten- 
tion to an important piece of national 
legislation in 1993 regarding the as- 
sessment of government performance, 
the Government Performance and Re- 
sults Act (GPRA). It analyzes the imple- 
mentation of this legislation in ten pilot 
sites. Again, the GPRA suggests a per- 
formance assessment model, and pin- 
points the potential and actual difficul- 
ties in applying it. 

The fifth article reviews the U.S. 
General Accounting Office’s effort to 
serve as a consultant to the Congress 
on how best to carry outjhe purposes 
of the GPRA. To play this role, the GAO 
studied performance-oriented assess- 
ment reform in four industrialized coun- 
tries. 

The sixth and seventh articles in the 
series discuss the implications of the 
entrance of government auditors in the 
U.S. and Europe into the performance 
measurement movement, and the prob- 
lem of neglecting evaluation research 
in the rush to tie budgeting to simple 
outcome measurement. 

The eighth and ninth articles give us 
a glimpse of the performance measure- 
ment movements in Australia and the 
Netherlands. 

The series concludes with two im- 
portant critiques of performance mea- 
surement systems, based on a study of 
their unintended effects. These articles 
suggest that we use caution in using 
such systems, and recommend a closer 
linkage between performance measure- 
ment and evaluation research. 



Performance 
Measurement and 
Continuous Quality 
Improvement 

Performance measurement and con- 
tinuous quality improvement seem 
welded together in the two-decade rush 
to focus on the results of social initia- 
tives, experiments and programs for the 
manifest purpose of producing more 
useful information for improving pro- 
gram effects over time. The relationship 
between these two concepts earmarking 
a new movement in social policy and 
program assessment on the one hand, 
and evaluation research on the other, 
has seldom been addressed directly. 

This issue is finally given some atten- 
tion in the June, 1995 edition of Evalua- 
tion Practice, a journal of the American 
Evaluation Association. The article is 
“Implications of Continuous Quality Im- 
provement for Program Evaluation and 
Evaluators” by Melvin M. Mark, a psy- 
chologist at Pennsylvania State College, 
and Edward Pines of the Industrial En- 
gineering Department of New Mexico 
State University. 

What is CQI? 

Unlike most analysts, Mark and 
Pines appear to equate continuous qual- 
ity improvement (CQI) with total quality 
management (TQM). Most experts on 
the contemporary performance mea- 
surement movement see TQM as a 
team-based management strategy for 
commiting decisionmakers to improving 
production processes and worker pro- 
ductivity, and for meeting customers’ 
expectations. TQM has also involved 
regularized monitoring of these process- 
es as well as of desired outcomes. CQI 
more often refers to the feeding back of 
various kinds of assessments of social 
programs to their management struc- 
ture, for making ongoing adjustments to 
the implementation, interventions, tar- 
get groups and/or outcomes of pro- 
grams. 

The emphasis in CQI is frequently on 
program implementation — i.e. organi- 
zational and service delivery structures 
and processes — and the information- 
fed back to management tends to come 
most often from a team cross-trained in 
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each other’s program areas and com- 
posed of program staff who have sought 
input from stakeholders. This contrasts 
with the feedback to managers coming 
from external evaluators. The general 
purpose of CQI is to identify quality 
problems and suggest solutions. While 
this purpose holds true for both TQM 
and CQI, these approaches are usually 
conceived as separate but complemen- 
tary efforts. 

The key to the potential relationship 
between evaluation research and CQI, 
the authors propose, lies in the fact that 
employer organizations and social pro- 
gram organizations committed to CQI 
are significantly different from tradition- 
al organizations. Those trained in CQI 
they say, exhibit these characteristics: 

■ They focus on customers’ internal 
and external needs and requirements. 

■ They are trained in, and continually 
practice systems analysis and strate- 
gies for identifying quality problems 
and program processes. 

m They use a team approach in study- 
ing, analyzing and carrying out im- 
provements in quality. 

■ They monitor the performance of 
processes in order to assist in the im- 
provement of quality, and in validat- 
ing that improvements in quality 
have in fact been made. 

These elements refer largely to the 
characteristics of CQI-oriented staff. 

The authors contend that CQI staff ex- 
pect and experience ownership in this 
organizational change effort, which dis- 
tinguishes them further from traditional 
organizations. However, a major compo- 
nent of CQI is commitment to this 
concept on the part of top-level man- 
agement. The overriding goal is im- 
provement of the quality of organiza- 
tional and service delivery processes, 
which is to result in improved services 
and outcomes. 

The authors claim that these facets 
of CQI set CQI-oriented organizations 
apart from traditional ones that tend to 
centralize management power, operate 
without prescribed procedures for iden- 
tifying or resolving quality problems, 
and emphasize results more than pro- 
cesses. The CQI organization requires a 
more participatory approach to manage- 
ment, organizational development, and 
service delivery. 
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Implications for Evaluation 
Research 

The authors propose that the partici- 
patory style of CQI-oriented organiza- 
tions makes evaluation planning easier 
to accomplish, and assists evaluators in 
implementing certain types of evalua- 
tion activities. For example, they say it 
aids evaluators in conducting evaluabili- 
ty assessments because the organiza- 
tion’s objectives and the processes 
through which they are meant to be 
achieved have been clearly defined. 

Also the design of the program (its the- 
ory of change), the resources available 
to the program, and the causal chain in- 
volved in moving clients from pre-inter- 
vention status to postprogram status 
tends to be specified. In this sense, a 
successful CQI-oriented organization 
comes close to precluding the need for 
an evaluability assessment, the authors 
suggest. 

What Mark and Pines refer to as in- 
ternal formative evaluation is also as- 
sisted in a CQI organization, they say, 
since in many cases the two approaches 
express the same goal. This goal is to 
provide relevant and timely information 
on the basis of which managers can 
make decisions — information that can 
influence the direction management de- 
cisions take. 

It may be a different story with sum- 
mative evaluations, they believe, be- 
cause of the potential for bias in draw- 
ing causal inferences from program 
results that have been produced sub- 
stantially differently, by a participatory 
CQI process versus a traditional one. 
Without a careful process evaluation 
(implementation study), they say, the 
validity and generalizability of outcome 
studies can be quite shaky. 

To prevent the production of biased 
results, Mark and Pines insist that CQI 
teams need training in evaluation re- 
search principles and standards, as well 
as cautions in interpreting the results of 
their efforts. But the largest challenge 
for evaluators may lie in convincing CQI 
staff that what they are doing, however 
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important and useful, is not the same as 
scientific evaluation, and does not come 
equipped with the same techniques for 
sorting out the cause-effect relationship 
between interventions, implementation 
modes, client characteristics and out- 
comes that an application of scientific 
method affords. 

The authors point to certain pro- 
grams that have been the ongoing 
object of CQI efforts, indicating that 
some of these programs are, at their 
core, inefficient and ineffective by eval- 
uation research standards and should be 
discontinued instead of "improved.” 

And CQI teams may mistake customers’ 
perceptions of increased quality for the 
real thing. The authors see a need to 
evaluate how well CQI approaches are 
working in terms of actually improving 
programs, using more rigorous evalua- 
tions to help determine their effective- 
ness vis-a-vis their stated objectives. 

And process evaluations of CQI-oriented 
organizations can be very valuable. Also 
they see the merging of CQl-generated 
information and information from pro- 
cess evaluations as a very useful mar- 
riage. 



Editor’s Note 



Readers may be interested in the 

following related references: 

Capper, C. A. and M. T. Jamison. "Let the 
Buyer Beware: Total Quality Management 
and Educational Research and Practice,” 
in Educational Researcher 22, 1983. 

Lipps, G. and P. R. Grant. "A Participatory 
Method of Assessing Program Imple- 
mentation,” in Evaluation Practice 13, 
1990. 

Wholey, J. S. “How Evaluation Can Improve 
Agency and Program Performance,” in 
Improving Government Performance ed- 
ited by J. S. Wholey, K. E. Newcomber 
and Associates. San Francisco, CA: Jos- 
sey-Bass, 1989. 
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Challenges in 
Developing 
Performance 
Measures and 
Standards: The Case 
of Educational Skills 

The skills standards project of the 
National Governors’ Association (NGA), 
conducted in cooperation with the Na- 
tional Center for Education and the 
Economy, is an important and useful il- 
lustration of some of the challenges in- 
volved in measuring performance. A 
series of reports on performance mea- 
surement have been produced by the 
project for use by the National Skills 
Standards Board created in the Goals 
2000: Educate America Act. This Legis- 
lation mirrors recommendations in the 
1990 report by the Commission on the 
Skills of the American Workforce. The 
long-term objective of the Board is to 
establish a framework for a national sys- 
tem of industry-based skills standards 
and credentials. 

In this article the focus is on one 
particular NGA report in the series, Per- 
formance Assessment Systems: Implica- 
tions for a National System of Skill Stan- 
dards. Volume II-Technical Report by 
John G. Wirt. Wirt carefully reviews 
several U.S. Department of Education 
projects, capturing insights for applica- 
tion in designing new systems of stan- 
dards-driven performance assessment in 
education. Figure 1 provides an over- 
view of the purposes of these projects. 

In all three projects the general mission 
has been to raise student learning stan- 
dards and reform assessment practices. 
The measurement concepts used are 
more “judgment-based” — similar to 
those used in occupational skills assess- 
ment — than "objective,” such as in ed- 
ucational testing. 

Varying Assessment Methods 

Wirt’s major interest is in the effica- 
cy of different versions of assessment 
systems. He claims that research has 
shown unequivocally that the use of tra- 
ditional standardized educational tests 
can narrow the learning environment by 
encouraging an overemphasis on factual 
knowledge and lower-level educational 
skills, even though standardization 
clearly increases the statistical reliability 
of performance measurement. Broaden- 
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Figure 1 ■ Overview of Certain U.S. Department of Education Projects 

Project Purposes 

National Board for Professional To improve the quality of education for all 

Learning children by setting national standards for 

accomplished teachers. 

To change the nature of teaching as a ca- 
reer. 

To foster ongoing professional conversa- 
tions about accomplished teaching. 

To influence the way the educational pro- 
fession views professional development. 

To change the public’s perception of teach- 
ing. 



New Standards Project To bring about systemic change in educa- 

tion through reforms in curriculum, as- 
sessment and teaching practices. 

To develop internally-benchmarked educa- 
tional standards all students should 
achieve. 

To create related performance assessment 
and examination systems. 

To replace “standardized, norm-refer- 
enced, multiple-choice tests of basic skills 
and knowledge that drive classroom prac- 
tice in the wrong direction” with new as- 
sessment forms based on the perfor- 
mance of tasks encouraging in-depth 
thinking and an integration of a range of 
skills. 



To develop a statewide performance as- 
sessment system supporting local school 
reform and improvement. 

To apply a series of curriculum “frame- 
works,” or standards, in instituting reform. 



California Learning Assessment 
System 



ing tests to incorporate higher levels 
and more complex reasoning skills forc- 
es a reduction in the level of standard- 
ization, making it difficult to generalize 
test results. Therein lies the dilemma in 
measuring complex skills. 

Wirt advocates for new forms of 
standards and assessment strategies. 
These are significantly different than 
standardized testing, testing based on 
the psychological model of skills and 
abilities, and the job-competency model 
based on industrial management con- 
cepts. 

In this respect, the three case exam- 
ples offer some common features of in- 
terest. Content standards are described 
as “texts” presenting a vision of expert 
performance rather than being extensive 
lists of discrete tasks or skills. Concrete 
examples of actual complex perfor- 
mance are used to communicate both 
content and performance-level stan- 

o 
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dards. Content standards distinguish 
among different “domains of thought 
and action” for which standards are to 
be set. 

All three assessment projects have 
relied on methods that present test-tak- 
ers with realistic problem situations and 
judge their responses using panels of 
raters who follow a structured method 
of evaluation. The measurement as- 
sumptions supporting this “interpreta- 
tive” model are substantially different 
from those underlying the statistical ap- 
proach. Wirt believes that the philo- 
sophical framework for setting stan- 
dards and assessing performance in 
these projects offers promising alterna- 
tives for industry skill standards and 
other systems. 

Problems in Setting Standards 

However, Wirt gives attention to two 
major problems in describing different 
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levels of performance and deciding 
which levels, or combinations of levels, 
should be used as absolute standards of 
expected performance. Setting these ab- 
solute standards is what gives standards 
the power to direct behavior. 

The first problem arises when the re- 
sults of assessments are used for “high- 
stakes, go-no-go” certification. In this 
case, the results of assessments must be 
unusually reliable or errors will surely 
be made. In fact, Wirt says there is am- 
ple evidence that such mistakes have 
been made. Another problem is that a 
single, overall judgment of pass/fail 
must be made in situations where there 
are multiple content dimensions for 
judging performance, multiple perfor- 
mance exercises for judging perfor- 
mance, and multiple judges. One solu- 
tion, Wirt suggests, is to create expert 
panels that develop performance stan- 
dards used mainly as benchmarks in the 
judging process. 

There is also the problem of general- 
izing the results of performance assess- 
ments . However, achieving comparabil- 
ity in judgments throughout a perfor- 
mance assessment system can be in- 
creased, Wirt proposes, by involving 
large numbers of practitioners in the 
judging process. This may reduce statis- 
tical reliability but it would increase 
comparability in complex performance 
systems. 

Comments on Wirt’s Analysis 

Although the analysis is education- 
specific, some of the general insights 
are applicable to the development of 
skills standards and the measurement of 
performance in workforce preparation 
systems as well as in other social pro- 
gram areas. The dilemmas and prob- 
lems identified in the educational field 
have transfer application to employment 
and training and other policy fields. An 
important caution can be borrowed 
from Wirt’s analysis, namely that a nar- 
row set of performance measures rarely 
describes the true complexity of interven- 
tions and outcomes adequately , even 
though the statistical reliability of per- 
formance judgments using these mea- 
sures — and their generalizability — 
may be greater. Consequently, in the 
context of growing reliance on perfor- 
mance assessment systems, there is a 
serious need to be vigilant about such 
problems and seek a better balance in 
developing them across all social policy 
initiatives. 
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Performance 
Measurement 
Systems and the 
National Performance 
Review 

In March 1993, President Clinton 
asked Vice-President Gore to lead the 
National Performance Review (NPR), an 
effort to “re-invent” the federal govern- 
ment. Annual reports on the progress of 
this initiative were expected. The first 
report of the NPR made no less than 
384 recommendations. Subsequent re- 
ports, and Accompanying Reports on 
special issues or particular agencies, 
added to these recommendations. At 
the same time, these reports gauged 
progress against NPR’s goals: to reduce 
the size and redefine the role of the fed- 
eral government. The underlying pur- 
pose was to make the central govern- 
ment more efficient and effective. 

In this article we summarize NPR 
recommendations regarding information 
production and performance measure- 
ment. One of the major actions taken in 
support of NPR’s agenda concerning 
these two issues is the development of 
performance agreements signed by the 
President with the directors of his major 
federal agencies. These agreements in- 
volve the development of measures of 
performance for each agency. By 1998, 
the federal government is expected to 
produce its first results-oriented finan- 
cial statement. In this sense, perfor- 
mance measurement and budgeting are 
to be closely coordinated for the first 
time. 

The 1993 NPR Accompanying 
Report Directed to the U.S. 
Department of Labor 

In the National Performance Re- 
view’s series of reports, twenty-one rec- 
ommendations were directed to the U.S. 
Department of Labor. Staff teams inter- 
nal to the Department were expected to 
participate with the NPR in implement- 
ing these recommendations. Although 
the suggestions focus mainly on cutting 
costs and red tape, consolidating select- 
ed programs, and integrating services, 
they also give attention to information 
needs and accountability issues. With 
respect to the consolidation of reem- 
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ployment programs, the recommenda- 
tions include developing more efficient 
and sophisticated data systems and col- 
lecting more adequate data for monitor- 
ing the progress of program participants 
and the “performance” of programs. Re- 
garding the creation of customer-orient- 
ed “One-Stop Centers” intended to pro- 
vide a comprehensive set of services 
across multiple federal and state pro- 
grams, the recommendations cite three 
kinds of information needs: 

■ Information accessible to customers 
on the labor market, and the quality 
of different education and training 
options. 

■ Information on the level and kind of 
satisfaction experienced by custom- 
ers with the program services they 
receive. 

■ Information about the extent to 
which programs are "performing” 
vis-a-vis the measures used to judge 
performance. 

The NPR views “ensuring public ac- 
countability” as an essential function of 
the federal government in the federal/ 
state/local partnership supporting the 
establishment and operation of the One- 
Stop Centers. An appropriate definition 
of accountability, NPR suggests, de- 
pends on acquiring sufficient informa- 
tion on the multiple programs involved, 
and on the development of adequate in- 
formation systems for the collection, 
storage, extraction and analysis of these 
data. 

To encourage states to develop state- 
wide workforce development systems 
and local workforce boards that more ef- 
fectively coordinate policy development, 
planning and oversight, the NPR recom- 
mends that federal funding to initiate 
One-Stop Centers be available only to 
states willing to increase accountability. 
NPR also suggests that the Department 
of Labor provide state workforce coun- 
cils with resources for capacity -building 
and information on “ best practices Per- 
formance-based competition across One- 
Stop Centers is also promoted. 

Strategic Planning 

The 1993 report essentially creates a 
new environment for national strategic 
planning regarding workforce develop- 
ment. The National Economic Council is 
expected to establish a strategic plan 
through the joint efforts of the Depart- 
ments of Labor, Education, Energy, 



Health and Human Services, and the Of- 
fice of Management and Budget — with 
the participation of the Departments of 
Agriculture, Commerce, Veterans’ Af- 
fairs, Housing and Urban Development, 
representatives of state and local gov- 
ernment, and the employer community. 
Program coordination is seen as a key 
task in developing a systematic, ongoing 
national planning process. A number of 
subtasks arc considered critical: 

■ Development of a core set of uniform 
terms and definitions across the myr- 
iad of workforce development pro- 
grams. 

■ Standardization of fiscal and admin- 
istrative procedures, rules and regula- 
tions. 

■ Development of a complementary set 
of results-oriented performance stan- 
dards. 

■ Collection of more useful qualitative 
information on program perfor- 
mance. 

NPR views these assignments as im- 
portant contributions to “a more com- 
prehensive, universal, customer-driven, 
integrated workforce development sys- 
tem.” To move closer to such a system, 
it is clear that agencies will need to vol- 
unteer considerably more staff time in 
the context of no new resources. 

The Development of a Core 
Set of Uniform Terms 
and Definitions 

This unique and difficult initiative is 
expected to produce substantial results: 

■ Joint intake forms. 

■ Common data elements. 

■ Sophisticated computer technologies 
to improve information, services and 
access. 

■ Comparisons across related pro- 
grams. 

In the 1992 JTPA Amendments, the 
Department of Labor was required to 
establish a federal interagency work 
group that would identify a core set of 
consistently defined data elements and 
definitions for JTPA, JOBS, the Perkins 
Act, and Title V of the Older Americans 
Act. 

The report of this task force, Core 
Data Elements and Common Definitions 
for Employment and Training Programs 
was published by the U.S. Department 
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of Labor in January 1996. (Issue #11 of 
Evaluation Forum reviewed this report.) 
This report also supports the creation of 
an interagency Workforce Development 
Council at the federal level. 

The Development of a 
Complementary Set of Results- 
Oriented Performance 
Standards 

A set of desired outcomes for most 
workforce development programs has 
already been identified, but the defini- 
tions of these outcomes often vary, and 
not all relevant outcomes are given at- 
tention. Furthermore, the quality of 
these outcomes is frequently ignored. 
More significantly, performance stan- 
dards may involve too few variables and 
measures, or fail to incorporate the most 
critical ones. Evidence indicates that 
standards, as a reward and sanction 
system affecting program funding, can 
redirect programs away from their in- 
tended goals and target groups. And 
standards may not require sufficient in- 
formation about the level of achieve- 
ment of key outcomes to be of maxi- 
mum fairness and use in planning and 
budgeting. 

The interagency work group is ex- 
pected to walk this mine field success- 
fully, developing a set of performance 
standards that represents the common 
desired outcomes for multiple programs , 
recognizing that not all programs 
should be guided by the same stan- 
dards, and that all standards require on- 
going adjustment based on evaluations 
and program experience. 

The Development of Qualitative 
Information on Program 
Performance 

Although the NPR report for the De- 
partment of Labor does not elaborate on 
what is meant by “qualitative” out- 
comes, we can assume that attitudinal 
measures of customer satisfaction are 
an example. The larger goal seems to be 
an expansion of the set of desired out- 
comes to include information about se- 
lected characteristics of these outcomes, 
such as the nature of the training re- 
ceived, the occupational category of the 
job recipients obtain, the nature of the 
work environment and the characteris- 
tics of the workplace. 



The 1993 NPR Accompanying 
Report on Mission-Driven, 
Results-Oriented Budgeting 

Tying the budgeting process to per- 
formance measurement, NPR defines 
the functions of budgeting broadly: 

■ A managerial process that determines 
the direction that should be taken in 
improving management and overall 
efficiency. 

■ A marketing process that helps pro- 
mote programs and services. 

■ An accountability process through 
which policy priorities and desired 
outcomes can be developed and 
monitored. 

Having so defined budgeting, the 
NPR concludes that traditional federal 
government budgeting has neglected 
goal-setting and overemphasized inputs, 
reducing government accountability. 

The NPR claims that a change in the 
culture of budgeting can "encourage 
long-term thinking, define goals, trans- 
late them into desire results, and use 
performance measures more effectively 
to make more informed decisions on 
program priorities and resource 
allocation . . . [and to] convert account- 
ability for spending money to account- 
ability for achieving results, within the 
resources allocated, to meet or exceed 
performance standards and objectives.” 

History 

Actually there is nothing new about 
the concept of results-oriented financial 
planning. It is standard in academic 
planning departments, as part of strate- 
gic planning principles. It was anticipat- 



ed early-on in the public sector — in 
the government performance movement 
of the 1940s and 1950s, the Program- 
ming , Planning and Budgeting frame- 
work emerging in the 1960s, the Zero- 
Based Budgeting of the 1970s, and the 
Management By Objectives and Total 
Quality Management concepts of the 
1980s. 

In the private sector, major corpora- 
tions restructuring for global competi- 
tion have reoriented their own priorities 
toward customer service, placing prima- 
ry emphasis on quality and requiring at- 
tention to continuous improvement in 
processes and products. Clearly bench- 
marking, or measuring performance 
against a standard, has come into its 
own. The public and private sectors are 
converging in embracing results. Which 
sector has had the most influence on 
the other is uncertain but a 1993 survey 
of corporate executives in 75 major 
American corporations, conducted by 
the U.S. TYeasury Department, conclud- 
ed that federal program managers 
should learn from “corporate best prac- 
tices,” referring to the growing interest 
in outcomes and their quality. 

Internationally, postindustrial coun- 
tries in Europe and Asia, also feeling the 
bite of broader competition, have in- 
creased their use of performance man- 
agement to guide public administration, 
program management and budgeting. 

We have entered a new era in which 
satisfying the taxpayer and the custom- 
er has moved to center stage. This is re- 
flected in the landmark Government Per- 
formance and Results Act of 1993. The 
purposes of the Act are outlined in Fig- 
ure 1. 



Figure 1 * Purposes of the Government Performance and Results Act 

■ To hold federal agencies accountable for achieving program results. 

■ To initiate a reform of performance through a series of pilot projects in federal agencies 
which test performance management concepts prior to full implementation of the Act — 
i.e. set goals, measure performance against these goals, and report publicly on pro- 
grams. 

■ To promote a new emphasis on results, on service quality, and on customer satisfac- 
tion. 

■ To require federal managers to plan how to meet objectives and provide them with in- 
formation about program results and service quality. 

■ To provide more objective information on achieving statutory objectives, and on the rel- 
ative effectiveness and efficiency of federal programs and spending. 

All agencies are to 1) define their long-term goals, 2) set specific annual performance tar- 
gets, and 3) report annually on the performance vis-a-vis the targets. 
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The interest in "re-inventing” gov- 
ernment does not necessarily mean that 
government has gone beyond the sheer 
rhetoric of the new accountability move- 
ment. A 1992 GAO report, Program Per- 
formance Measures: Federal Agency Col- 
lection and Use of Performance Data , 
indicated that a survey of federal agen- 
cies and their departments showed that 
fewer than half had actually used per- 
formance information in monitoring 
their progress in achieving agency goals, 
even though two-thirds had developed 
strategic plans and were collecting per- 
formance data. Even fewer were using 
performance measures in reports dis- 
seminated to other agencies and the 
public. More significantly, few were us- 
ing performance information in making 
policy and management decisions. The 
existence of NPR set off a siren for 
change. 

NPR's Recommendations 

Acknowledging that unacceptably 
high federal deficits have spurred a 
reconsideration of the budgeting para- 
digm internationally, the NPR is never- 
theless insistent that its recommenda- 
tions can apply equally well in a period 
of abundance. These recommendations 
fall into several categories: 

■ The strengthening of accountability 
for results, with politicians defining 
the political priorities, followed by 
agreements with managers about 
goals and how achievement of these 
goals is to be measured. 

■ A number of postindustrial countries 
are now requiring the development 
of mutually agreed-upon missions, 
goals and objectives for organization- 
al performance, which tie the perfor- 
mance of senior government officials 
to a government’s major goals. Many 
corporations are doing the same, 
linking performance agreements with 
organizational goals and team perfor- 
mance. In the U.S., effectively imple- 
menting the Government Perfor- 
mance and Results Act is a case 
example of a much broader trend. 

■ Once decisions have been made and 
agreements established, the empow- 
ering of managers to achieve their 
goals, by providing sufficient resourc- 
es and reducing excessive regulatory 
burdens. 







■ The streamlining and improving of 
the budget development process, to 
provide managers with more timely 
information on policy priorities and 
funding levels, and to coordinate 
budget resources with agency mis- 
sions, goals and results. 

These three initiatives were in sync 
with the Government Performance and 
Results Act in 1) developing strategic 
plans, 2) establishing performance 
agreements, and 3) creating results-ori- 
ented task forces to coordinate the de- 
velopment and use of performance mea- 
sures, ensuring that performance 
information was reliable and valid. But 
the NPR's recommendations went fur- 
ther: 

■ Each agency is to compare well with 
other similar organizations in achiev- 
ing "benchmark standards for quali- 
ty, service and cost.” 

■ All directors, managers and budget 
offices are to incorporate perfor- 
mance objectives and results as key 
management elements in developing, 
reviewing and deciding on budgets. 

■ Appropriation accounts and line 
items are to be aligned with the pro- 
grams designed to carry out policies, 
and are to be charged with the costs 
these programs generate. 

■ Agency heads are to adjust their op- 
erating plans to clarify performance 
goals. 

■ The President is to propose biennial 
budgets , and Congress is to adopt bi- 
ennial budget resolutions and appro- 
priations. 

■ The Administration is to establish a 
coordinated team approach and de- 
velop an internal mechanism for al- 
locating funding according to broad 
government missions, agency by 
agency. 

Such suggestions are tied to the 
NPR’s major interest in results, quality 
and customer service, with the logical 
chronology of steps similar to these: 

■ Development of an agency’s mission. 

■ Clarification of an agency’s goals and 
objectives, consistent with its mis- 
sion. 

■ Description of the strategies the 
agency plans to use in achieving the 




goals and objectives, drawing on pol- 
icies and program legislation, histori- 
cal and current program data, and 
the needs and expectations of the 
program’s customers and stakehold- 
ers. 

■ Identification of the kinds and quan- 
tity of goods and services to be pro- 
duced and for whom, and the quality 
expected in those goods and servic- 
es. 

■ A comparison of programs with simi- 
lar ones, or with examples from the 
private sector, to establish perfor- 
mance benchmarks, based on the ac- 
complishments of other programs. 

■ Development of operational plans 
and shorter-term performance targets 
or standards for reaching the bench- 
marks, based on evaluations of cur- 
rent activities and efforts to improve 
them. 

■ Establishing effective agency teams 
to support the above, and to oversee 
the monitoring of progress. 

■ Development of common goals for 
data collection across programs, such 
as common outcome measures and 
benchmarks across programs with 
similar purposes, interventions and 
target populations. 

■ Inclusion of performance objectives 
and results as key elements in finan- 
cial and managerial reviews. 

Challenges in Implementing 
NPR's Recommendations 

Although there is now considerable 
agreement in government that it is es- 
sential to measure and monitor perfor- 
mance, the development of performance 
measures and standards, and particular- 
ly the notion of linking these with bud- 
geting, pose some problems. NPR is re- 
freshingly frank about them. 

Performance measurement needs to 
be simple and useful, the NPR report 
recognizes, with the goal of using a 
small number of easy to define mea- 
sures. Developing performance mea- 
sures is to involve living with a particu- 
lar set of assumptions about what is 
most important to achieve, and then 
making choices among competing long 
vs. short-term goals. This is to be ac- 
complished in the context of the cost of 
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acquiring good information about the 
achievement of those goals and the hu- 
man resources needed to interpret that 
information for policy use. 

Cost and resource considerations can 
narrow the set of variables utilized. Not 
all critical elements of a program may 
be measured, risking an insufficient un- 
derstanding of its full complexity. Fur- 
thermore, the measures chosen may not 
be reliable or valid. Also different feder- 
al programs have significantly different 
desired outcomes, therefore perfor- 
mance must be measured differently in 
diverse programs. 

Unlike previous accountability 
thrusts, the new movement requires 
considerable customer feedback in de- 
veloping performance measures. Admi- 
rable and useful as this may be, it can 
slow and complicate the process. It 
sometimes risks divisiveness that reduc- 
es an acceptance of the measures and 
the planning process. And the move- 
ment to focus more exclusively on re- 
sults may create “performance informa- 
tion bureaucracies” that develop a life 
of their own. The NPR does not ignore 
such challenges. 

The 1995 NPR Report 

This time the NPR report was in 
book form, written by the Vice-Presi- 
dent — a journalist before moving into 
politics. This most recent report is an 
interesting review of progress in imple- 
menting previous recommendations, but 
it does not neglect ways in which the 
federal agencies should resolve continu- 
ing problems. Gore reiterates that there 
was a tendency in the past for the feder- 
al government to collect information on 
inputs to the neglect of outcomes , and 
with little attention to the quality of ei- 
ther. He claims that agencies often for- 
got what their initial missions and goals 
were, in the context of lax expectations 
about understanding the results of their 
efforts. 

Gore reminds us that the 1993 Gov- 
ernment Performance and Results Act 
was a way to guide agencies away from 
a focus on procedures and processes to- 
ward an emphasis on performance. The 
Act required agencies to define their 
goals more precisely, make specific 
commitments to achieving them, and 
establish formal “outcome measurement 
systems” to allow them to track 
progress against goals. 



It also requested that ten pilot 
projects be developed by the federal 
agencies to test the utility of this new 
legislation. Since 1993, no less than sev- 
enty projects have been created. For the 
first time, twenty-four major federal 
agencies have been required to provide 
annual audited financial statements . 

The Act essentially set the stage for 
the National Performance Review . But 
Gore points out that the NPR was not 
the only effort at government reform in 
the 1990s. He reviews a host of legisla- 
tive initiatives that provide additional 
evidence of a new awareness of the im- 
portance of “government accountabili- 
ty,” such as the Government Manage- 
ment Reform Act , the Chief Financial 
Officers Act , and the Federal Acquisi- 
tions Streamlining Act. And other forces 
have been at work simultaneously, at 
both the federal and state levels, which 
supported NPR’s reinvention strategy — 
such as a growing interest in studying 
customer satisfaction. NPR built on this 
interest by establishing Customer Service 
Standards, which amounted to perfor- 
mance standards specific to customer 
satisfaction. The standards involved 
specific outcomes and commitments for 
achieving them. 

A parallel effort developed bench- 
marks for service quality, drawing 
insights from private/public “best prac- 
tices.” To lend strength to the accumula- 
tive effect of different reform efforts, in 
March 1995 President Clinton issued a 
directive reinforcing the requirement 
that agencies develop and publish cus- 
tomer standards. He required them to 
survey customers to identify their 
needs, measure customer satisfaction 
based on what outcomes mattered most 
to the customers themselves, and inte- 
grate customer service activities with all 
other performance initiatives. Among 
other purposes, this approach was ex- 
pected to increase managers’ ability to 
continuously improve program organiza- 
tions, services and outcomes. 

What the NPR was attempting to do 
was to infuse government with what it 
felt was the best version of private sec- 
tor concepts and practices , to improve 
its capability, responsiveness, and per- 
formance. The public had become the 
stockholders, the government the corpo- 
rate manager, program services the 
product, and performance measurement 
the way to monitor market share, sales 
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levels, earnings growth and share pric- 
es. This cross-cultural transfer required 
that government organizations under- 
stand as clearly as successful businesses 
the goals toward which activity should 
be directed. Now, in 1996, federal agen- 
cy-White House performance agree- 
ments identify each agency’s perfor- 
mance goals and objectives, document a 
commitment to honor them, and seek to 
reward senior agency personnel on that 
basis. 

The linguistics of the NPR and relat- 
ed efforts to reform government is rife 
with downsizing, streamlining, restruc- 
turing and privatizing. Government ac- 
tion taken since 1993 and reported by 
the NPR would suggest that a great deal 
has been accomplished by this combi- 
nation of strategies. Gore compares the 
NPR approach to enterprise develop- 
ment, suggesting that a good enterprise 
measures what matters, and these mea- 
sures tell managers how and in what 
ways they are or are not succeeding as 
well as what to do to improve perfor- 
mance. “If the essence of enterprise de- 
velopment is risk-taking and experimen- 
tation,” Gore says, “measurement tells 
us how the experiment is going.” 

Some Concluding Review 
Comments 

The National Performance Review 
has maintained faithfulness to its mis- 
sion, to recommend and monitor the 
creation of a new form of central gov- 
ernment. The principles and standards 
it seeks to promote are becoming the 
“universals” for postindustrial govern- 
ments. What now seems to be needed, 
as part of NPR’s activities, is a perma- 
nent task force to study the risks and 
benefits of moving in the results-oriented 
direction, giving careful attention to the 
potential risks and monitoring the 
tradeoff over time. 

This task force should also address 
the issue of what role evaluation re- 
search should play in this new move- 
ment. Science-based program evaluation 
is mentioned in passing, and not nega- 
tively, but it is not made an integral part 
of the performance movement. Without 
evaluation research, we will not know 
the policy significance of the “results” 
we have so compulsively measured and 
reported in the 1990s. 
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Editor’s Note 



The National Performance Review 
has published yearly progress reports 
based on its mission and goals, and a 
large number of Accompanying Re- 
ports . 

Annual Reports 

September 1993: From Red Tape to Results: 
Creating a Government That Works Bet- 
ter and Costs Less 

September 1994: Creating a Government 
That Works Better and Costs Less: Sta- 
tus Report 

September 1994: Putting Customers First: 
Standards for Serving the American Peo- 
ple. 

September 1995: Common Sense Govern- 
ment: Works Better and Costs Less, by 
Al Gore. 

Accompanying Reports 

Report for the U.S. Department of Labor 
Transforming Organizational Structures 
Improving Customer Service 
Mission-Driven, Results-Oriented Budgeting 
Reinventing Human Resources 

Management Reengineering Through Infor- 
mation Technology 

Rethinking Program Design 

Strengthening the Partnership in Intergov- 
ernmental Service Delivery 

Additional References of Interest 

Epstein, P “Get Ready: The Time for Perfor- 
mance Measurement is Finally Coming”’ 
i n Public A dm inistra ti on Review, S e pte m - 
ber/October 1992. 




Kamensky, J. and J. Tessauro. Performance 
Measurement in the United Kingdom and 
Australia. Paper presented at the Perfor- 
mance Measurement Seminar. Washing- 
ton, D.C.: National Academy of Public 
Administration, May 1992. 

Kettl, D. Improving Government Perfor- 
mance: An Owner's Manual. Washington, 
D.C.:The Brookings Institution, 1993. 

Moore, M. H. Accounting for Change: Rec- 
onciling the Demands for Accountability 
and Innovation in the Public Sector. 
Washington, D.C.: Council for Excellence 
in Government, 1993. 

Oregon Benchmarks: Standards for Measur- 
ing Statewide Progress and Government 
Performance. Oregon Progress Board, 
December 1992. 

Osborne, D. and T. Gaebler. Reinventing 
Government : How the Entrepreneurial 
Spirit is Transforming the Public Sector. 
Reading, MA: Addison-Wesley, 1992. 

Schick A. Five Reforms in Search of Budget 
Control: Congress Versus the Federal 
Budget. Washington, D.C.: Congressional 
Research Service, January 1992. 

U.S. Department of Treasury: Performance 
Measurement: Report on a Survey of Pri- 
vate Sector Performance Measures. 
Washington, D.C., January 1993. 

U.S. General Accounting Office: Managing 
for Results: Experience in Four Europe- 
an Countries and How They Might Be 
Applied in the U.S. Washington, D.C., 
January 1993. 

. Using Performance Measures 

in the Federal Budget Process. Washing- 
ton, D.C., July 1993. 

Volcker, P. A. Leadership for America: Re- 
building the Public Service. Washington, 
D.C.:The Federal Reserve, 1989. 
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Testing the 
Government 
Performance and 
Results Act 

The National Academy of Public Ad- 
ministration (NAPA) was chartered by 
the U.S. Congress in 1967 to improve 
the art and science of public gover- 
nance. It is composed of four hundred 
current and former federal Cabinet of- 
ficers, members of Congress, governors, 
mayors, state legislators, jurists, public 
managers, business executives and aca- 
demicians. In 1994, NAPA formed an 
Advisory Panel on Improving Govern- 
ment Performance. This panel was co- 
chaired by Harry Hatry of The Urban 
Institute and Joseph Wholey of the Uni- 
versity of Southern California. Wholey 
is now at the President's Office of Man- 
agement and Budget (OMB). The results 
of the panel’ s work were reported in a 
monograph published in 1994, Toward 
Useful Performance Measurement: Les- 
sons Learned from Initial Pilot Perfor- 
mance Plans Prepared Under the Govern- 
ment Performance and Results Act. 

In the preceding article about the 
President’s National Performance Re- 
view, we discussed the Government Per- 
formance and Results Act of 1993 
(GPRA). This legislation requires a pro- 
cess of goal setting, performance mea- 
surement and regularized review as a 
basis for a new performance-oriented 
approach to public governance — all in- 
gredients in a new definition of strategic 
planning. Ten pilot projects were to test 
this approach over a three year period 
to provide insights for implementing the 
Act. However, no less than seventy-one 
performance plans were submitted to 
OMB. This involved thirteen of the four- 
teen federal departments. Fifty-two 
plans were accepted. Quite remarkably, 
this occurred in the context of minimal 
OMB guidelines, no training, and no ad- 
ditional funding for this large project. 

NAPA had taken a strong interest in 
performance measurement since 1979 
when it published the Productivity Im- 
provement Handbook for State and Lo- 
cal Government. Subsequently the orga- 
nization passed a resolution supporting 
performance measurement and monitor- 
ing. Later NAPA established the Adviso- 
ry Panel to review the performance 
plans of the pilot projects testing the 
GPRA, and to synthesize ideas from this 
unprecedented planning effort. This ar- 
ticle summarizes NAPA’s review. 
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Review Criteria 

NAPA reviewed forty of the perfor- 
mance plans for pilot projects within a 
particular set of assumptions about per- 
formance measurement. The Panel’s 
perspective was that the measurement 
of program “outputs” and “outcomes” 
provides essential information about 
goal achievement — that is, about prob- 
lems needing resolution and how to im- 
prove programs. Performance data were 
also seen as establishing public account- 
ability. The review of the projects 
looked for strategies being used that 
would test these assumptions. 

Therefore, the Panel was careful to 
specify criteria for judging the plans 
submitted by the pilot projects. The 
main criteria were: 

■ The reasonableness, clarity and com- 
prehensiveness of mission state- 
ments, and the relationship of mis- 
sions to performance plans. 

■ The mix of performance indicators 
suggested in the plans, such as mea- 
sures of inputs, processes, outputs 
and outcomes, as well as efficiency 
and service quality measures — but 
with the main emphasis on out- 
comes. 

■ The adequacy of plans for judging 
data reliability and validity. 

■ Whether the use of numerical perfor- 
mance targets based on performance 
indicators were suggested, and what 
the basis was for these targets. 

■ The likely utility of the performance 
measures and targets suggested in 
plans to policymakers and program 
managers in improving program per- 
formance, and to Congress and the 
public in demonstrating program ac- 
countability. 

The Results of the Panel’s Review 

Applying these criteria, the Panel’s 
review produced these conclusions: 

Timing. Too little time was made 
available to federal departments and 
agencies for preparing performance 
plans. Personnel in the pilot projects 
clearly needed training in performance 
measurement, as had been recommend- 
ed in the Act. 

Scope. Some agencies had avoided 
developing performance indicators be- 
cause they felt they had little control 
over the influences to be measured, and 
that performance based on targets 
would not explain the extent to which 
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observed outcomes could be attributed 
exclusively to programs. Interestingly, 
agency officials proposed that only in- 
depth program evaluations were capable 
of establishing this kind of cause-effect 
relationship, and therefore only when 
evaluations were linked with perfor- 
mance management could they feel 
comfortable about the proposed new 
approach. 

Organizational responsibility for 
performance measurement. Participa- 
tion in developing performance mea- 
sures and targets by all levels of agency 
management — particularly by top-level 
administrators — appeared lacking. 

This weakened agencies’ commitment 
to performance plans. Also, few agen- 
cies had given thought to performance 
measurement regarding their relation- 
ships with other agencies with similar 
missions. 

Relationship between performance 
indicators, legislative goals and agen- 
cy missions. There was insufficient 
linkage between the measures devel- 
oped and the legislative purposes, mis- 
sions, and strategic plans of agencies. In 
some cases, mission statements were 
entirely too broad to reveal these con- 
nections; in others no strategic plan was 
evident. There was a general tendency 
to rely on existing administrative data in 
developing measures, with too little em- 
phasis on what measures might be best 
in monitoring agency activities against 
their missions and goals. Few agencies 
had involved their clients in providing 
ideas about performance measures. 

Nature and complexity of the per- 
formance measures. Likely due to the 
lack of guidelines by OMB, agencies dif- 
fered in their definitions of performance 
measures. Also, the sets of measures 
were frequently narrow and limited, re- 
peating the earlier focus on process 
data. This was so even though most 
agencies were collecting outcome data. 
The lack of customer feedback was evi- 
dent in the paucity of measures for 
judging customer satisfaction. 

Validation of data sources. Few per- 
formance plans included methods for 
assuring that the data collected for per- 
formance management purposes were 
reliable and valid. Where these issues 
were addressed in plans, only simple au- 
dit strategies, rather than scientific 
methods, were suggested. 

Improving program performance. 

In reviewing plans for how performance 
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information was to be communicated to 
program managers for program im- 
provement, a major purpose of the 
GPRA, the panel found plans unclear 
about the basis for selecting perfor- 
mance targets. In many instances, no 
targets were proposed, in others the tar- 
gets chosen appeared not to have in- 
volved a re-evaluation of traditional ap- 
proaches. Most serious, there was little 
evidence that performance information 
would actually be used to improve per- 
formance — i.e. would be disseminated 
to operating units of programs. Further- 
more, most plans did not suggest disag- 
gregating performance data — such as 
by type of clients, organizational unit, 
or geographical area. 

Explaining program performance. 

Plans did not propose ways to include 
explanatory information in program re- 
ports that would suggest potential rea- 
sons for a particular level of perfor- 
mance on measures and targets. 

Dissemination of performance in- 
formation to the Congress and public. 

Performance measures and targets were 
often presented in plans without an ad- 
equate context in terms of the level of 
understanding of Congressional leaders 
and the public. The larger design of 
programs, of which performance mea- 
surement was a part, was not well com- 
municated or explained in commonly 
understood language. 

The Panel’s Recommendations 

The Panel actively considered recom- 
mendations for improving performance 
plans. Figure 1 (page 58) summarizes 
their main suggestions. 

The initial report card on the pilot 
projects does not translate to an “A,” 
but much progress has been made in a 
very short time by large-scale govern- 
ment bureaucracies that historically re- 
sist change. Given the time line, it sure- 
ly was not realistic for OMB or NAPA to 
have expected more at this juncture. 
Meanwhile, the lessons gleaned in the 
NAPA review will contribute important 
insights for next steps in this long- 
term, complex and promising federal 
project. The GPRA requires all federal 
agencies to participate in strategic plan- 
ning and performance measurement by 
fiscal 1999. Performance targets are to 
be provided by September 1997 and an- 
nual performance reports are to begin 
by March 2000. This is a very ambitious 
planning agenda. 
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Figure 1 ■ NAPA Advisory Panel’s Recommendations 
Issue Recommendations 



Timing and Scope of the Performance 
Plans 


All departments and agencies should speed the development of of the strategic plans, the 
development of performance measures, and the identification of sources of information to 
support the measures. 

Training in developing performance measures for program improvement purposes needs to 
be provided, as well as opportunities for sharing information on this task across federal de- 
partments. 

Providing explanatory information about the probable relationship between programs’ inter- 
ventions and their performance should be an integral part of performance reports , so that 
outcomes likely attributable to programs can be distinguished from those likely due to other 
factors. The limitations of performance measurement should be made explicit. 


Departmental Responsibility for 
Performance Measurement 


The top level of departments and agencies should play a more active role in strategic plan- 
ing and performance measurement, particularly in formulating and improving performance 
management systems. 

Program managers should play a strong role in identifying performance measures that will 
help them improve programs. And the federal government should develop a method for 
studying the range of agency and program performance measures, to identify and reconcile 
interrelated missions and measures. 


The Use of Performance Measurement 


Consistent linkages should be evident between missions, goals, strategic plans, and annual 
performance plans — and these processes and products should be connected logically to 
the broader goals of top management and the Congress. 


Management Systems to Achieve 
Legislative Goals, Agency Missions, 
and Program Improvement 


Each agency should establish a hierarchy of goals and performance measures appropriate 
to the different levels of government, which are internally consistent. 

If new data collection appears essential, agencies and programs should propose the addi- 
tional measures needed. 

In the process of developing measures, agencies and programs should seek input from their 
organizational stakeholders and clients which can be incorporated in performance measure- 
ment systems. 

Annual performance targets should be set realistically, but should be developed to encour- 
age progress beyond traditional levels. Consequently, performance plans should include 
long-term performance goals as well as the targets for the following year. The definition of 
"excellent performance” needs to be incorporated. 


The Nature of Performance Indicators 


Top agency management should encourage a focus away from the exclusive use of process 
indicators to the use of a broad range of performance indicators. 

A consistent set of measures, their operational definitions, and guidelines for applying them 
should be classified usefully, such as by performance indicators , performance targets, in- 
puts, outputs, outcomes and resources required. 

Performance plans should identify basic data sources and what methods will be used to as- 
sure that measures are valid in representing key factors, are reliably collected, and are time- 
ly. Technical assistance needs to be provided to maintain the quality and relevance of auto- 
mated performance-based information systems. 


Reporting Performance 


Classifying information will assist different groups of organizational users at the federal level 
in fully utilizing performance reports. 

Simple diagrams can clarify program operations, and illustrative cases can help users un- 
derstand the program context. 
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Editor’s Note 



For more information on perfor- 
mance measurement issues, please 
see these publications: 

Hatry, H. et al. Performance Measurement - 
A Guide for Local Elected Officials. 
Washington, D.C.iThe Urban Institute, 
1981. 

Hatry, H. and C. Wye. “Evaluation — It’s Po- 
tential Utility for Operating Managers,” in 
Organization Excellence edited by J. 
Wholey. Lexington, MA: D.C. Heath and 
Co, 1987. 

Hatry, H. “Determining the Effectiveness of 
Government Services,” in Handbook of 
Public Administration edited by H. Perry. 
San Francisco, CA: Jossey-Bass, 1989. 

Hatry, H. and D. Fountain. Service Efforts 
and Accomplishments Information: Its 
Time Has Come. Norwalk, CT: Govern- 
mental Accounting Standards Board, 
1990. 

Hatry, H. and D. Fisk “Measuring Productivi- 
ty in the Public Sector,” in Public Produc- 
tivity Handbook edited by M. Holzer. New 
York, NY: Marcel Dekker, Inc., 1992. 

Hatry, H. et al. “Eleven Ways to Make Perfor- 
mance Measurement More Useful to 
Public Managers,” in Public Manage- 
ment, September 1994. 

Wholey, J. and J. Newcomer. Handbook of 
Practical Program Evaluation. San Fran- 
cisco, CA: Jossey-Bass, 1994. 



The Experiences of 
Other Countries with 
Performance 
Measurement 

The U.S. General Accounting Office 
(GAO) has produced a series of reports 
on management reforms that involve 
performance measurement. This article 
concentrates on Managing for Results: 
Experiences Abroad Suggest Insights for 
Federal Management Reforms , published 
in May, 1995. The report responds to a 
Congressional request for ideas about 
implementing and assessing the man- 
agement reforms required by the Gov- 
emment Performance and Results Act of 
1993. It analyzes the experience of four 
industrialized countries recognized as 
leaders in the movement toward "re- 
sults-oriented” government reform — 
Australia, Canada, New Zealand and the 
United Kingdom. 

The Reform Concept and 
Its Trade-Offs 

The major features of the reforms 
studied in these countries by the GAO 
are strategic planning , the development 
of goals and objectives, the measure- 
ment of progress in meeting these ends, 
and the reporting of progress to the pub- 
lic. Although “management by objec- 
tives” thinking is not in any sense new, 
many Americans caught up in our cur- 
rent performance measurement move- 
ment may be unaware that the four 
countries examined have been carrying 
out results-focused management chang- 
es for over a decade. This effort has par- 
alleled their pursuit of changes in the 
organizational culture of private sector 
entities. 

It is not surprising that one of the 
most challenging tasks in instituting 
performance management systems in 
these countries has been changing the 
managerial culture. There have been 
substantial differences in the speed with 
which the desired cultural changes have 
occurred, and in the extensiveness of 
the changes across diverse governmen- 
tal departments and agencies within 
these countries. 

In all four nations, the reforms were 
catalyzed by serious economic problems 
and pressures to provide better evidence 
of goal achievement, the same influenc- 
es fueling our own managerial revolu- 
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tion. The associated need to provide 
more useful information about the effi- 
ciency and effectiveness of government 
departments, agencies and programs 
moved reforms away from a focus on 
inputs and processes, and in the direc- 
tion of outcome measurement. 

As a tradeoff for increased account- 
ability through outcome measurement, 
typically managers in these countries 
have been given greater flexibility and 
autonomy in allocating resources in 
newly decentralized and devolved social 
service environments. However, the re- 
forms bind regional and local govern- 
ment organizations to central govern- 
ments through multi-level performance 
agreements. These assure that managers 
will be held accountable for the results 
they achieve, based on agreed upon per- 
formance goals. In these agreements, 
the definitional steps from mission to 
goals to objectives to measures of perfor- 
mance are to be a logical progression, 
each step leading to increasingly more 
specific understandings of the meaning 
of efficiency and effectiveness. 

There has been some confusion 
among managers, however, about the 
difference between devolution , which 
the GAO defines as the transfer of deci- 
sionmaking capacity from higher orga- 
nizational levels to lower, and decentral- 
ization, which the researchers define as 
the redistribution of functions from cen- 
tral organizational units to more dis- 
persed ones. Some managers have de- 
fined the new flexibility in both 
directions, irrespective of the intent of 
the reforms. 

Some Useful Conclusions from 
the GAO Study 

The GAO researchers fotind these 
characteristics to describe the move- 
ment toward results-oriented manage- 
ment in the four countries studied. 

■ Development and definition of a 
logically consistent set of depart- 
ment, agency and program missions, 
goals, objectives and performance 
measures. 

■ Holding management accountable 
for desired results. 

■ Provision of greater management au- 
tonomy in allocating resources with- 
in overall budget ceilings, and in de- 
veloping incentives for meeting 
goals. 

■ Capacity-building and technical as- 
sistance to support results-oriented 
management. 



67 BEST COPY AVAILABLE 



59 





Evaluation Issues and Activities ■ The Performance Measurement Movement 



■ Development of information systems 
to collect, report and analyze depart- 
ment, agency and program data for 
monitoring and assessment purpos- 
es. 

Participatory Planning and 
Implementation 

In the United Kingdom and Australia, 
management and staff are sometimes 
brought into the strategic planning pro- 
cess, leading to a greater sense of own- 
ership of goals and a stronger commit- 
ment to improving performance. Annual 
operating plans translate strategic plans 
into detailed objectives for day-to-day 
activities. In developing such plans, 

New Zealand and the United Kingdom 
have emphasized the more immediate 
measures of performance — i.e. mea- 
sures of quantity, quality, efficiency and 
“output” cost — rather than longer-term 
effectiveness measures. But all four gov- 
ernments use a combination of qualita- 
tive assessments, customer surveys and 
more formal evaluations to estimate the 
effects of programs. There is a strong 
sense of fairness in protecting managers 
from being blamed for outcomes heavily 
influenced by economic conditions and 
other phenomena beyond their ability to 
alter. 

Mixed-Method Assessment 

In Australia and Canada, both perfor- 
mance measurement and program evalu- 
ation are viewed as integral parts of 
management, a perspective we still 
struggle to create in the U.S. In Austra- 
lia, departments are required to evaluate 
the effectiveness of each of their pro- 
grams every three to five years. In Cana- 
da also, departments are expected to be 
capable of evaluating programs for their 
effectiveness. Both “ongoing effective- 
ness measurement” for monitoring pur- 
poses and “periodic studies of program 
effectiveness” are central to the new re- 
forms. Monitoring involves collecting 
and reporting information on a set of 
outcomes; evaluation involves a more 
in-depth review of the fit between pro- 
gram objectives and outcomes. The use 
of comparison or control groups is rec- 
ommended, but program evaluations 
more often rely on surveys of program 
recipients, particularly regarding cus- 
tomer satisfaction. 

Feedback from Internal 
Reviews and Levels of 
Management 

The GAO researchers’ review of gov- 
ernment evaluations and their inter- 
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views with officials suggested the fol- 
lowing: 

■ Performance measures need to flow 
directly from a program’s objectives, 
irrespective of whether they repre- 
sent short-term or longer-term ef- 
fects. 

■ Performance measurement systems 
should involve a small set of key 
measures — i.e. those considered 
critical to performance. 

■ Measures should represent outcomes 
that realistically can be achieved by 
a program’s management. 

■ Program staff should play a role in 
identifying measures. 

■ The range of performance measures 
should assure balance, to preclude 
an overreliance on any one measure. 

■ Performance systems should include 
both qualitative and quantitative in- 
formation, on both process and out- 
comes. 

■ Performance results should be ex- 
plained, to the extent possible, not 
simply reported. 

■ Performance systems should provide 
more detailed information on the 
achievement of measurable objec- 
tives regarding factors considered 
outside the control of program man- 
agement as well as influences viewed 
as being under management control. 

■ Performance information should be 
made available to the public and to 
all levels of the managerial hierarchy. 

Performance Measurement 
Activities 

The United Kingdom and Canada 
measure the quality of services and re- 
port results to the public on a regular 
basis. All four countries utilize perfor- 
mance agreements between different 
governmental levels which incorporate 
performance targets. And all four re- 
quire departments to make performance 
reports available to their central govern- 
ments and the public. The GAO found, 
however, that the use of these reports 
by central governments has been mini- 
mal, and that efforts are underway to 
increase utilization. 

In these countries, managers were 
being given much more freedom to 
make their own spending decisions and 
develop their own incentive systems for 
rewarding high performance. More of 
this kind of flexibility was, in turn, be- 



ing provided to lower levels of manage- 
ment. Private sector “market” concepts 
and strategies were being introduced to 
create a more competitive interdepart- 
mental, cross-program culture, rein- 
forced by financial and other incentives. 
It is the GAO’s conclusion that the ap- 
plication of these market-oriented ideas 
has, in fact, made government activity 
more efficient and effective. But formal 
evaluations indicate that there are risks 
involved in increasing managerial au- 
tonomy, such as a loss of control by 
central governments, and it is clear that 
top management has in some cases only 
halfheartedly cooperated with the re- 
forms. 

Supports for Performance 
Systems 

Meanwhile, long-term investments in 
information systems and training have 
supported reform. In all four countries 
there has been a recognition that high- 
er-level management needs to assist 
lower-level managers in understanding 
the value of the reforms and in imple- 
menting them — in particular strategic 
and operational planning, performance 
measurement and budget flexibility. 

Performance Reporting and 
Utilization 

The United Kingdom and Canada 
routinely develop and publish perfor- 
mance targets and standards for depart- 
ments and agencies directly serving the 
public, and report achievements to the 
public. The United Kingdom’ s Citizen 
Charter and Canada’s Service Standards 
Reforms require agencies to publish ser- 
vice standards, measure performance 
against these standards, and request 
feedback from the public. Australia and 
New Zealand place less emphasis on 
providing customers with program in- 
formation and encourage less customer 
feedback, but they include many service 
quality measures in their performance 
systems. These systems also involve rec- 
ommendations for resolving perfor- 
mance problems. 

The use made of performance re- 
ports has been uneven, however. The 
Canadian Parliament found in 1992 that 
department performance reports did not 
provide the kinds of information most 
needed by decisionmakers, did not link 
outcomes with costs, and did not sum- 
marize and interpret performance clear- 
ly. The lack of staff, time and expertise 
often eroded the utility of performance 
information to potential users. 
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Some Comments 

This GAO study reinforces the con- 
clusions of other expert reviews of the 
performance measurement movement in 
the industrialized countries, namely 
that there are positive benefits in in- 
creasing government attention to the 
short-term and longer-term outcomes 
and effects of departmental, agency and 
program activity. However, there is little 
information yet about the effects of in- 
creasing managerial flexibility and in- 
corporating other kinds of incentives for 
changing the managerial culture. And 
the issues of what form performance in- 
formation should take for dissemination 
purposes, to whom it should be distrib- 
uted, how full utilization of this infor- 
mation can be assured, remain thorny. 
Clearly utilization is a major issue, since 
none of these reform efforts will bear 
fruit without it. 



Editor’s Note 



Additional GAO reports on perfor- 
mance management issues are: 

Program Performance Measures: Federal 
Agency Collection and Use of Perfor- 
mance Data, May 1992. 

Performance Budgeting: State Experiences 
and Implications for the Federal Govern- 
ment February 1993. 

Improving Government : Measuring Perfor- 
mance and Acting on Proposals for 
Change, June 1994. 

Managing for Results: State Experiences 
Provide Insights for Federal Management 
Reforms. December, 1994. 

Government Reform: Goal-Setting and Per- 
formance. March 1995. 
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Performance 
Auditing and 
Evaluation Research 



Evaluation researchers are beginning 
to feel the invasion of performance au- 
ditors into professional territories they 
assumed were off limits to anyone with- 
out research training. Not many re- 
searchers were aware of the 1990 
report, Service Efforts and Accomplish- 
ments Reporting: Its Time Has Come, 
produced by the Government Account- 
ing Standards Board (GASB). The Board 
is a major player in developing guide- 
lines and setting standards for the au- 
diting and accounting profession. 

This report encouraged auditors to 
move into the measurement of out- 
comes at the public agency level, re- 
garding the organization as a whole 
and/or its programs. What the GASB re- 
port was all about was performance au- 
diting, an entirely new specialization 
within the fiscal auditing world. This 
auditing function clearly overlaps with 
evaluation research in a period in which 
performance measurement is being 
launched enthusiastically. 

Driven by new economic scarcities 
that were focusing attention on govern- 
ment size, waste, abuse and inefficien- 
cy, the performance measurement 
movement had begun in the late 1970s 
as the stress on greater public account- 
ability came into its own. This move- 
ment was energized by stricter account- 
ability-oriented legislation at the federal 
and state levels, and the other fiscal 
containment efforts in the 1980s. There 
was a public groundswell for wanting 
more detailed and sophisticated infor- 
mation on the results flowing from the 
expenditure of tax dollars. 

Performance auditing essentially in- 
volves the conducting of descriptive 
gross outcome evaluations by auditors 
untrained in social research principles 
and methods. A performance audit ac- 
cepts limited, general managerial expla- 
nations of why and how the gross out- 
comes observed are occurring, but the 
clear emphasis is on a small group of 
outcomes without attention to the rela- 
tionship between interventions and out- 
comes, or without much concern about 
which outcomes can be attributed ex- 
clusively to a government agency or 



program rather than to other influences 
and forces. The outcomes selected are 
not necessarily those selected by an 
agency or program manager or an eval- 
uator, since they emerge from a fiscal 
auditing, not a program-knowledgeable 
environment. 

Most problematic is the tying of per- 
formance audits to budgeting decisions. 
This has opened a new Pandora’s Box, 
where fatal social remedies and organi- 
zational efforts can easily result in more 
rather than less funding based on mis- 
leading information or misinterpreta- 
tions of the data available. 

The GASB report suggests an ideal- 
type chronology of steps which not only 
equate auditors with evaluators but re- 
duce both professionals to simple out- 
come measurement: 

■ Policymakers authorize or re-autho- 
rize programs and provide funding. 

■ Management collaborates with poli- 
cymakers, customers, employees and 
taxpayers to develop expected out- 
comes and measures of outcomes. 

■ Independent auditor performs finan- 
cial/legal compliance audits. 

■ Internal/external auditors perform 
program evaluations and verify re- 
porting of service efforts and accom- 
plishments information. 

■ Policymakers identify, improve or 
discontinue programs that have out- 
lived their usefulness, cease to make 
policy sense, can be run more effi- 
ciently, or have a lower priority than 
others. 

The growth of the performance mea- 
surement movement in the 1980s and 
1990s has strengthened the welding of 
the results of outcome studies of all 
kinds with major budgeting processes 
and decisions. In this process the role of 
the auditing community has grown 
stronger. And in a period of vastly re- 
duced government resources, budget 
deficit concerns, and anti-big-govern- 
ment sentiment, it looks far less expen- 
sive and more efficient to conduct 
quick-turnaround performance audits 
than longer descriptive outcome studies 
by researchers and full blown net im- 
pact studies. But the risks of producing 
a narrow, biased information base, and 
overrelying on its use in allocating re- 
sources are increasingly more serious. 
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Editor’s Note 



For a more detailed look at distinc- 
tions between performance auditing 
and evaluation research, please see: 

Chelimsky, E. “Comparing and Contrasting 
Auditing and Evaluation,” in Evaluation 
Review 9, 1985. 



More Insights About 
Performance 
Auditing and 
Evaluation Research 



Another perspective on the relation- 
ship between evaluation research and 
auditing is provided by an interesting 
interview in the October 1995 issue of 
Evaluation. A new international evalua- 
tion journal published by Sage and edit- 
ed by Elliot Stern of the Tavistock Insti- 
tute in London, its emphasis on “theory, 
research and practice” has successfully 
reduced the isolation of American eval- 
uators from applied researchers in other 
countries, broadening their professional 
approaches and expanding their re- 
search repertoires. 

This issue of Evaluation Forum re- 
ports on an interview conducted by 
Stern, an evaluation researcher, with 
Alan Pratley, Financial Controller of the 
European Commission (EC) and Acting 
Director General of its Financial Control 
Directorate. The interview is a case 
study of how auditors and evaluators 
see the world differently, talk past one 
another, hold to their territorial views, 
and are sliding into competition with 
one another in a period in which “ac- 
countability” is an all-important priority 
of governments. 

In his introduction to the interview, 
Stern points to the growing emphasis on 
both public sector evaluation and audit- 
ing in all the European Union countries. 
His comment is that government audit- 
ing “has been at the cutting edge of 
many contemporary regulatory and 
public management initiatives." He 
claims that as early as the mid-1980s, 
the evaluation research and auditing 
professions were borrowing concepts 
from each other, which has contributed 
to the management concepts being for- 
mulated under increasing public pres- 
sure to reform public management in- 
ternationally. By the 1990s, Stern says, 
the U.S. government and many other 
central governments were pushing hard 
for the institutionalization of perfor- 
mance management and evaluation, 
stronger linkage between evaluation and 
auditing activities, and the integration 
of evaluation results with budgeting de- 
cisions. 
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In a frank and accurate assessment, 
Stern also notes the tendency for gov- 
ernment bureaucracies, in the midst of 
the new accountability movement, to 
focus only on certain categories of eval- 
uative and financial information. Such 
data are being viewed as the most rele- 
vant information for measuring the ben- 
efits of policies and programs to cus- 
tomers, stakeholders and society, when 
in fact this information and the meth- 
ods for collecting it are not always 
consistent with the scientific and pro- 
fessional standards of evaluation re- 
searchers. On the other hand, Stern sees 
the nearly worldwide performance man- 
agement/budgeting drive as opening up 
new opportunities for evaluators, de- 
spite its sizable challenges. 

Increasing Financial Control 

Pratley describes the considerable re- 
sponsibilities of the European Commis- 
sion's Financial Controller in terms of a 
150-person staff processing 360,000 
transactions per year. This staff has the 
right and duty to monitor how all Euro- 
pean Commission funds are spent, one 
of the largest subfunds being the Struc- 
tural Funds for national, regional and 
local human service, community and 
economic development projects in the 
EC's member countries. However, only 
since 1990 has the Financial Controller 
been assigned the task of carrying out 
an ongoing, regularized program of in- 
ternal audits. 

With respect to the Structural Funds, 
member states are required to develop 
adequate management and control sys- 
tems. The EC audit division has the 
right to access information for EC audit- 
ing purposes, and the ability to require 
that member states perform their own 
audits if problems are identified. The 
work of the EC audit office is overseen 
externally by the European Court of Au- 
ditors, which is in turn monitored by 
the European Parliament. 

Pratley refers to the new amendment 
to the Financial Regulations (introduced 
in 1991) as a significant change in EC 
attitudes toward the auditing function. 
This amendment required the develop- 
ment of "quantified objectives” and the 
monitoring of the extent to which they 
were achieved in terms of efficiency and 
cost-effectiveness. Prior to this legislative 
revision, the Financial Controller had 
concentrated on inputs and processes 
rather than outcomes and impacts. This 
reflected the pre-1990s experience of au- 
ditors internationally. 
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Financial Assessment Methods 

In Stern’s interview, Pratley is asked 
to explain the assessment strategies 
now used to audit progress against 
quantitative measures of performance. 
Regarding the Structural Funds, which 
cover, among other things, innovative 
training programs, infrastructure devel- 
opment projects and rural improvement 
projects, Pratley sees outcomes as being 
fairly easily measured. But Stern presses 
him further, based on his own experi- 
ence in evaluating Structural Funds pro- 
grams. He suggests that there is a ten- 
dency to develop performance measures 
only for the easy-to-quantify outcomes, 
while the more important outcomes ex- 
pected of social initiatives are often in- 
tangible, such as increased innovative- 
ness, coordination, and 
competitiveness. 

Pratley’s response is sensitive to this 
problem. However, he sees great benefit 
in requiring projects to describe more 
precisely what population is involved, 
what areas are covered, and what their 
activities and outcomes are — and de- 
manding that these projects incorporate 
monitoring and evaluation strategies 
within initial and ongoing project plan- 
ning. 

Stern also asks about potential collu- 
sion between the interests of the leader- 
ship of the EC and the internal auditing 
division, related to the need to avoid 
negative audit results in the context of 
scarce funding to continue the Structur- 
al Funds and other projects. Pratley 
views more rigorous evaluations as the 
purview of external experts. He is not 
concerned that program managers may 
mask their true level of achievement in 
an effort to preserve problematic pro- 
grams. The EC financial audit division 
counters this possibility, he says, by em- 
phasizing with managers that problems 
encountered in programs are not neces- 
sarily due to poor management. This at- 
titude, he feels, reduces managerial anx- 
iety and assures accurate reporting. 

Also Pratley feels that the kind of 
monitoring currently carried out by the 
EC focuses appropriately on an adher- 
ence to the general framework and prin- 
ciples developed by the auditing divi- 
sion rather than on specific guidelines 
— particularly the general commitment 
to cost-effectiveness and ongoing moni- 
toring and evaluation. This general em- 
phasis, he says, gives managers suffi- 
cient leeway to produce trustworthy, 
useful information. 



Stern deftly moves Pratley toward 
the Controller’s current definition of 
“evaluation,” by asking for a response 
to what Stern has absorbed in his own 
work: 

So far you have identified 
evaluation with financial 
control and its concern with 
sound management, the ef- 
ficient use of resources, and 
achieving results. What is 
specific about evaluation? 

Pratley distinguishes between the 
new EC model and the traditional 
French system in which obedience to 
rules is the main priority in financial 
auditing. Now the emphasis is on pro- 
viding evidence that you arc doing what 
you proposed to do for the lowest possi- 
ble cost, or that you are applying cost- 
effectiveness principles to what you are 
doing and you have quantified your ob- 
jectives and tracked results against 
them. This, Pratley claims, is the mean- 
ing of “evaluation” within an auditing 
framework: making a judgment about 
goal achievement. To institutionalize 
this concept of evaluation, Financial 
Control is developing “the performance 
audit.” 

The Isolation of Evaluation 
Research from Performance 
Auditing 

As an evaluator, Stern cannot leave 
the discussion at the point Pratley wish- 
es to end it. He comments that evalua- 
tion researchers are as interested as 
new-age auditors in studying results. 

But they are also concerned, he says, 
about studying implementation process- 
es that help explain results. 

Objectives are not simply 
the benchmark against 
which results are judged — 
how they evolve and change 
is also a focus of attention. 

Pratley sees this broader and more 
holistic interest of evaluation research- 
ers satisfied by ongoing, if quite gener- 
al, feedback from managers about con- 
flicting objectives and implementation 
problems, rather than from formal ef- 
forts by internal auditors or outside ex- 
perts to determine why and how pro- 
grams go astray or succeed. The 
Monitoring Committees created in mem- 
ber states to oversee Structural Funds 
programs, for example, would be more 
appropriate bodies, he says, for identify- 
ing process (implementation) issues and 
difficulties. [Please see the article in the 



Evaluation Issues and Activities section 
of this issue on the European Commis- 
sion guide for Monitoring Committees], 
The EC policy directorates, other than 
the auditing division, he believes, 
should be the sector collecting this kind 
of information from managers, encour- 
aging them to adjust their implementa- 
tion approaches based on such informa- 
tion. That does not mean, Pratley 
recognizes, that Financial Control 
should ignore implementation issues in 
conducting performance audits. 

Stern continues this line of thought, 
however, asking why the goal of finan- 
cial auditing should not be to determine 
the critical relationship between what 
the results are and what appears to ex- 
plain them, particularly in large-scale 
programs where one needs to know 
whether results represent a policy or a 
program failure. But Pratley clearly 
leaves this kind of evaluation to other 
divisions within EC or to external evalu- 
ators. 

Stakeholder-Oriented Auditing 
and Evaluation 

Stern poses another interesting ques- 
tion, namely how the new EC financial 
management framework will accommo- 
date the emerging emphasis on stake- 
holder oriented evaluative approaches — 
where those with a clear and significant 
investment in a program are brought 
into evaluation planning, implementa- 
tion and utilization. Pratley’s response 
is an intriguing non-answer. Substantial 
concern across member countries of the 
European Union about past misuse and 
potential abuse of EC funds has led the 
EC to concentrate more strenuously on 
the way funds are used, according to 
Pratley, and particularly how cost 
effectively they are used. Focusing at- 
tention on fraud and abuse serves to 
send a message to stakeholders that the 
EC is serious about an issue of interest 
to them. Pratley answers Stern’s ques- 
tion about stakeholder involvement in 
the evaluative process by indicating that 
this issue is satisfied by the narrow re- 
sponse to stakeholder worries about 
abuse. 

Later in the interview it becomes 
clear that Pratley finds it important to 
distinguish between the formal role of 
evaluation research units within the pol- 
icy directorates of the EC, and the finan- 
cial control division. The former, he 
assumes, should conduct process, out- 
come and impact evaluations when 
needed to judge more comprehensive 
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longer-term efficiency and effectiveness. 
Consequently these units could address 
a broader range of interventions, imple- 
mentation structures/functions, and re- 
sults. Performance audits were not to go 
beyond a general judgment of efficiency 
and cost-effectiveness a la money man- 
agement practices, utilizing a set of bu- 
reaucratically-identified (rather than 
stakeholder-responsive) outcomes. 

Stern ultimately asks the most pene- 
trating question for evaluators: to what 
extent are the worlds of evaluation re- 
searchers and performance auditors 
competitive, and even in potential con- 
flict? In so asking, Stern acknowledges 
that the two professional territories are 
conceivably at war in some European 
Union countries. Again, the Financial 
Controller sidesteps this issue by claim- 
ing that an active intolerance is required 
toward managers in the EC directorates 
and member countries who wish to 
place their own organizational agendas 
ahead of obedience to the new perfor- 
mance audit framework. This says a 
great deal about how willing the finan- 
cial wing of the EC is to work with 
managers, internal evaluation units, and 
external evaluators in a collaborative, 
stakeholder-sensitive way. 

However, Pratley does indicate that 
Financial Control and the Directorate 



General of the Budget have jointly spon- 
sored a study of communities using a 
team of nine experts in different fields 
— a more collaborative model — which 
will provide information to all sectors of 
the EC. He also wants a contact for his 
division developed within each internal 
evaluation unit or a liaison in director- 
ates that do not yet have such units. 

Some Thoughts About the 
Interview 

In summary, what Stern evokes in 
this interview between professionals 
with substantially different training and 
experience is an interplay that reveals 
the perennial differences in conceptual- 
ization, interpretation, professional 
principles and methods, and scope of is- 
sues between those trained in social sci- 
ence research and those trained in ac- 
counting and financial management. 
Meanwhile, two new directions are evi- 
dent in the European Union, as in the 
U.S. and other countries: 1) the move- 
ment of auditors into performance mea- 
surement and interpretation, earlier the 
exclusive domain of evaluation re- 
searchers, and 2) the close linking of 
performance auditing with budget deci- 
sions which have far reaching effects on 
policies and the program strategies de- 
signed to carry them out. 



Comments on the 
Performance 
Measurement 
Movement in 
Australia 

Sue Funnell gives us another interna- 
tional perspective on performance mea- 
surement in her paper about the Austra- 
lian approach to performance issues 
prepared for the International Panel on 
Performance Measurement in Vancou- 
ver, Canada in November, 1995. Funnell 
is a researcher with the firm Perfor- 
mance Improvement Limited in Huntleys 
Point, Australia. Her presentation drew 
not only interested evaluators in the Eu- 
ropean Union, North America and Asia 
but also some inspector generals. The 
question period did not avoid debate 
about the appropriateness of auditors 
moving into the assessment of perfor- 
mance, but there was comfortable 
agreement that the pressure to measure 
performance was driven by economic 
forces impinging on the governments 
represented. 

In most of the postindustrial coun- 
tries, international economic competi- 
tion and associated fiscal pressures have 
led to an increased emphasis on man- 
agement, benchmarking, quality assur- 
ance, outcome measurement and contin- 
uous improvement processes . There has 
been a convergence across these nations 
even in the language used to describe 
these functions, tasks and processes. As 
with all these countries, the decentrali- 
zation of money flow and programs has 
placed greater emphasis on accountabil- 
ity at lower levels of government. 

Consequently the need for these lev- 
els to feed back information to central 
governments for resource allocation 
purposes has grown. Understandably, 
central governments have increased 
their demand for program outcome in- 
formation as they cope with a loss of 
control over program design and imple- 
mentation. In the new social policy cli- 
mate, countries have sought to balance 
concern about the value of social pro- 
grams to their recipients with increased 
cost efficiency as organizations. This 
has led to linking performance measure- 
ment to budgeting . 
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Australia is a useful case example. 
Funnell reports that performance mea- 
surement has taken several forms in 
Australia, remarkably similar to trends 
in the U.S. and the European Union: 

Benchmarking, focusing on out- 
comes or best practices. The main is- 
sues have been the comparability of the 
data used to develop benchmarks, the 
potential for the misuse of data, and the 
controversy over whether benchmarking 
should focus more on end results based 
on quantitative data than on informa- 
tion about the nature of implementation 
processes. There is concern that govern- 
ments will prefer quantitative bench- 
marking exclusively, which Funnell be- 
lieves could raise confidentiality issues 
and be misleading as a criterion for 
budgeting decisions. 

Total Quality Management, focusing 
on the measurement of the quality of 
management processes. The key issue, 
according to Funnell, is whether private 
sector concepts can be applied appro- 
priately to public sector management, 
which must deal with more ambiguous 
definitions of efficiency and effective- 
ness. 

Quality Assurance, focusing on the 
assessment of the quality of organiza- 
tional processes and procedures, often 
using a set of quality standards. Impor- 
tant issues are whether the use of stan- 
dards is appropriate, how relevant these 
standards are, and whether they actual- 
ly improve quality. Clearly there are 
problems with these assessment strate- 
gies, mainly in terms of objectivity, cov- 
erage of important factors, and research 
expertise. 

The Australian Inspector-General 
conducts periodic reviews of the quality 
of performance information. A review in 
1993 provided these conclusions about 
the information base: 

■ The quality of program management 
tends not to be related to program 
outcomes. 

■ The measurement of management 
quality is often based on unstated or 
questionable assumptions about the 
link between program strategies, re- 
sources, outcomes and performance. 

■ The quality of information about 
management does not provide a 
good basis for assessing management 
or program performance. 



■ Most information about cause-effect 
relationships is based on monitoring 
data rather than on more rigorous 
evaluations. 

Customer Satisfaction, focusing on 
“customer service guarantees” or the 
quality of the exchange between service 
deliverers and service receivers. Funnell 
emphasizes several issues. One is what 
qualities of the above exchange are best 
to measure. Some facets of this relation- 
ship may not be directly related to the 
outcomes of service delivery. Also, at- 
tention to client satisfaction may lead to 
a neglect of the needs of families, other 
groups, and the community. Further- 
more, some of the measures of custom- 
er satisfaction may be so superficial that 
they lose their utility in getting at more 
critical customer satisfaction variables. 

Continuous Quality Improvement, 

focusing on information that can im- 
prove practice — both processes and 
their impact. Australia is currently using 
a hierarchy of six stages of organization- 
al development as a framework for 
studying continuous improvement in 
agencies and programs. A concern is 
that agencies must be motivated to en- 
gage in activities consistent with the 
continuous improvement concept, and 
this is difficult in the context of repeat- 
ed budget cuts and the devaluation of 
the role of government and its accom- 
plishments — a trend occurring in all 
the postindustrial countries. 

Australia not only emphasizes the 
forms of performance measurement out- 
lined here, but has a formal mandate 
and timetable: 

■ Assessments of the need for certain 
programs must be made. 

■ Annual performance reports are re- 
quired for all programs. 

■ Evaluations of all programs are to be 
conducted on a cycle of 3-10 years, to 
determine their appropriateness, effi- 
ciency and effectiveness. 

■ The central government publishes a 
register of completed evaluations 
twice a year. For each evaluation, the 
register summarizes its purpose, 
methodology, and major findings. 

Evaluations are expected to be an on- 
going management responsibility, and 
are an integral part of budgeting and 
planning. They are management-driven, 
emphasizing how resources have been 
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used and with what effect. Rather than 
simply measuring gross outcomes, the 
focus is on net impact, but adequate 
methodologies for determining the latter 
are not being used due to the lack of 
funds and research expertise. 

Funding for evaluation has been sub- 
ject to the same emphasis on cost effi- 
ciency as other government functions, 
so there are only minimal funds for 
evaluation and these generally go to in- 
ternal evaluation units. In most cases, 
agencies conduct self-evaluations with 
existing staff, although some evaluation 
and performance measurement training 
is provided despite central government 
ambivalence about the expense. Select- 
ed agencies are able to use third-party 
independent evaluators. 

A more general issue identified in the 
Funnell paper is the fact that formal 
support for more expert program evalu- 
ations is decreasing at the central gov- 
ernment level in Australia, which paral- 
lels the trend elsewhere. The fear is that 
the critical linkages between inputs, 
processes, outputs and impacts will be 
neglected. Consequently the outcomes 
being measured may be assumed to tell 
the government what the exclusive re- 
sults of a given program’s interventions 
are, when the information simply does 
not support such conclusions. 

Excessive emphasis on measurable 
outputs with minimal attention to net 
impacts is an obvious danger cited by 
Funnell. Also there are gaping holes, 
she says, in the information collected by 
agencies regarding the size and charac- 
teristics of target groups, the nature of 
target group needs, and the extent and 
features of service delivery. To address 
these concerns, a number of govern- 
ment-sponsored performance measure- 
ment activities are underway, particular- 
ly the collection of performance indi- 
cators in nine areas to compare their 
ability to measure efficiency and effec- 
tiveness — police, courts, corrections, 
schools, vocational education, hospitals, 
aged/disabled services, family/child 
welfare, and public housing. And mea- 
surable performance standards are be- 
ing established which reflect the level 
and quality of services. 
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Comments on the 
Performance 
Measurement 
Movement in the 
Netherlands 



The interest in performance mea- 
surement is now an international obses- 
sion, fueled by common economic prob- 
lems throughout a global market. It is 
also a correlate of decentralization and 
privatization, which have transferred 
accountability to lower levels of govern- 
ment and private sector service deliver- 
ers. The Netherlands is a case in point. 

In this brief article we review a con- 
ference paper by Frans L. Leeuw pre- 
pared for the Panel on International 
7 fends in Performance Measurement at 
the International Evaluation Conference 
in Vancouver, Canada in November 
1995. Leeuw is professor of sociology at 
the University of Utrecht and serves on 
the Netherlands Court of Audit in the 
Hague. 

Accountability Strategies 

The central government of the Neth- 
erlands is guided fiscally by a Budget 
and Accounting Act which requires 
government ministers to carry out sev- 
eral kinds of periodic assessments: 
financial management audits; perfor- 
mance measurements; and policy and 
program evaluations . The Netherlands 
Court of Audit is an independent body 
that performs assessments for the cen- 
tral government. Unlike the United 
Kingdom, there are no comparable local 
government offices. 

Leeuw’s analysis focuses singularly 
on central government performance 
measurement and evaluation regarding 
the use of the following array of inter- 
ventions (see Figure 1). Leeuw’s thesis 
is that information on the efficiency and 
effectiveness of these policy tools is mini- 
mal 

In 1992, the central government de- 
cided to require more information on 
outputs and effects, but a subsequent 
survey indicated that only fifty percent 




of budget expenditures could be ex- 
plained based on outcome data. Conse- 
quently the Ministry of Finance required 
that annual budgets provide evidence of 
the contribution outcome data on pro- 
grams had made to an understanding of 
particular budget items. Despite this 
more specific directive, Leeuw’s conclu- 
sion from analyzing central budget in- 
formation is that government agencies 
were continuing to resist the use of per- 
formance measures in interpreting ex- 
penses. In addition, the central govern- 
ment had been slow to institutionalize 
policy and program evaluations, even 
though academics have urged that it do 
so for two decades. 

The “Performance Paradox” and 
Its Unintended Consequences 

Leeuw sees the “proliferation and 
noncorrelation” of performance mea- 
sures as a new paradox to be addressed. 
He points out that performance mea- 
sures tend to be only weakly correlated. 
He also feels there are too many of 
them. Beyond that, he claims the in- 
creasing use of performance measures 
to monitor programs is leading to the 
growth of “second order performance 
assessment” by external auditors and fi- 
nancial analysts rather than analyses by 
evaluation researchers. And he suggests 
that performance measurement can 
have other negative effects: 

■ It can inhibit innovation and lead to 
organizational paralysis. 

■ It can lead to an excessive reliance 
on performance standards. 

■ It can focus attention too exclusively 
on the quantifiable aspects of perfor- 
mance. 



Figure 1 ■ Leeuw’s List of Policy 
Interventions 

Subsidies 

Quangos 

Public information campaigns 

Government-sponsored enterprises 

Contracts 

Permits 

Inspectorates 

Legislation 

Loans and guarantees 
Levies 

Fees and charges 



Response from Attenders at the 
Performance Measurement Panel 

The international conference was 
jointly sponsored by a number of Euro- 
pean and Asian evaluation research as- 
sociations, yet a large contingent of au- 
ditors attended. This was clearly related 
to the movement of auditors into the 
performance audit field, which is gain- 
ing credibility world-wide and begin- 
ning to crowd evaluation research as the 
strategy of choice in determining gov- 
ernment and program performance. 

The response to Leeuw’s concerns 
about the downside of performance 
measurement were well received. The 
central tendency in the audience was to 
worry about the rush to quick judg- 
ments of government operations and 
programs via a narrow set of perfom- 
ance measures and standards, without 
sufficient attention to broader theoreti- 
cal measurement and methodological is- 
sues in determining efficiency and 
effectiveness. The auditors present were 
not so much concerned about these is- 
sues, but rather about the need for addi- 
tional staff trained in performance au- 
diting and more funding to add such 
staff. But the strong trend toward study- 
ing the results of government activity 
was everywhere evident. 
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Improving 
Performance 
Systems in 
Supporting Program 
Goals 

The first systematic performance 
measurement system for social pro- 
grams was the performance standards/ 
incentive system developed for the fed- 
eral Job Training Partnership Act (JTPA) 
programs authorized in 1982. Legislat- 
ing the use of standards and a mecha- 
nism for assuring their application was 
driven in large part by perceived weak- 
nesses in the previous Comprehensive 
Employment and Training Administra- 
tion (CETA) program. Because of the de- 
centralized and privatized nature of 
JTPA, the program involves an applica- 
tion of standards at the federal level for 
states, at the state level for regional Ser- 
vice Delivery Areas (SDAs), and at the 
local level for service providers. States 
are allowed to adjust standards for sub- 
state areas based on measures of certain 
variables felt to be major factors outside 
the control of State Job Training Coordi- 
nating Councils that can influence their 
performance. Up to 5% of program 
funds flowing to states are to be used to 
reward SDAs for complying with or ex- 
ceeding standards, or to provide under- 
performing SDAs with technical assis- 
tance intended to raise their level of 
desired outcomes vis-a-vis standards. 

A number of studies have examined 
the effects of using such a system. In a 
1985 study for the U.S, Department of 
Labor, Katharine Dickinson and Richard 
West found that states emphasizing the 
goal of exceeding standards spent more 
funds on each terminee. However, 
spending per client entering employ- 
ment was no higher. A 1993 study by a 
doctoral student at Stanford University, 
Michael Cragg, found that the areas re- 
ceiving the greatest rewards were enroll- 
ing individuals with more work experi- 
ence. However, the more experienced 
individuals had a lower gain in earn- 
ings. Additional studies suggested that 
SDAs were resisting enrolling their quo- 
ta of welfare clients due to the fear of 
failing to “perform” adequately. 

These and other studies suggested 
that performance standards might not 
be enhancing the achievement of the 
program’s intended goals. On the other 



hand, the use of a less formal perfor- 
mance system in the Massachusetts ET 
Choice program, and in the GAIN pro- 
gram in Riverside, California — both 
employment and training programs — 
indicated that clear-cut performance 
goals and incentives did improve goal 
achievement. 

The PropositionThat 
Performance Standards 
Systems “Work” 

The potential positive effects of per- 
formance standards are the focus of a 
June, 1995 monograph published by W. 
E. Upjohn Institute for Employment Re- 
search, Using Performance Indicators to 
Improve the Effectiveness of Welfare-to- 
Work Programs. The author, economist 
Timothy J. Bartik, details the history 
summarized at the beginning of this ar- 
ticle, placing emphasis on the use of 
performance management systems for 
welfare reforms. Although the current 
national welfare program, JOBS, was to 
be reinforced by a performance system . 
similar to that used in JTPA, Bartik ex- 
plains that the federal government post- 
poned recommending performance 
measures for JOBS until October 1996, 
and performance standards until Octo- 
ber 1998. This is long past the presiden- 
tial election which will surely direct the 
form future welfare reform initiatives 
will take. The manifest reason for the 
delay, he says, was sensitivity to the 
possibility that outcome-based perfor- 
mance measures and standards have the 
potential for being misleading indicators 
of program goal achievement. 

Bartik refers the reader to the Octo- 
ber, 1994 report of the U.S. Department 
of Health and Human Services which 
summarizes research by the Manpower 
Demonstration Research Corporation on 
state welfare reforms. Differences in av- 
erage outcomes across these initiatives 
often showed no correlation with goal 
achievement. Within states, there was 
often variation across different counties. 
Sometimes the correlation between 
standards and goal achievement was ac- 
tually negative. And program outcomes 
within counties were often greater for 
the less disadvantaged. 

Meanwhile, in a 1995 study the U.S. 
General Accounting Office (GAO) con- 
tended that the delay in developing 
standards for JOBS was damaging this 
national program. Managers were not 
focusing enough attention on the em- 
ployment of clients, a major goal of 
JOBS. Linkages with employers re- 
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mained weak. Managerial energy was 
still narrowly focused on reporting the 
number and type of clients participating 
in JOBS activities. Bartik points out that 
the federal government literally did not 
know who got jobs or left the welfare 
system on an annual basis. 

Leaning towards the GAO’s position, 
Bartik’s thesis is that “it is feasible to 
develop good indicators of performance 
for a particular welfare-to-work pro- 
gram, office or contractor.” He believes 
that performance indicators can moti- 
vate key actors to be more effective in 
achieving a program’s true goals, and 
can provide information about “best 
practices.” To accomplish this, he pro- 
poses that performance measurement 
systems must be simple, timely, and 
control for factors other than the pro- 
gram that influence whether or not wel- 
fare clients achieve the goals intended 
for the program. 

Key program goals, he proposes, are 
improved self-esteem, better job skills, 
employment, earnings gains, and re- 
duced welfare dependence. Simple pre- 
post outcome evaluations will not sur- 
face such improvements, he says. Ex- 
perimental net impact studies can do 
so, but they are often costly and time- 
consuming, and are not likely to be 
conducted on a regular or widespread 
basis. Also, the latter raise ethical ques- 
tions about withholding interventions 
from eligible clients. The advantage of 
performance indicators and standards, 
Bartik says, is that their use does not re- 
quire such rigorous evaluations. The 
downside, however, is that they may 
distort the intended value of a program. 
But Bartik sees light at the end of that 
tunnel in the form of "better perfor- 
mance indicators” and “careful use.’’ 

Better Indicators and More 
Appropriate Utilization 

Bartik suggests a number of cautions 
in developing more effective indicators 
for performance management systems: 

■ Performance indicators must be 
linked to information on the strate- 
gies of particular program organiza- 
tions and staffs so that we know why 
a particular program component is 
successful, [This sounds like a rec- 
ommendation to explore all of these 
aspects of programs: organizational 
elements, implementation strategies, 
program interventions and out- 
comes.] 

■ Indicators need to be timely and un- 
derstandable. 
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■ It must be easier to increase perfor- 
mance by achieving a program's in- 
tended goals than by selecting clients 
who will help managers meet perfor- 
mance standards. 

■ Program managers must be motivat- 
ed to use performance measures in al- 
locating resources. 

■ Indicators should be simpler and less 
accurate as one gets closer to the indi- 
vidual staff level, and more statisti- 
cally sophisticated at higher levels of 
the welfare system. [The reason Bar- 
tik gives is that “it is easier at the 
staff level to substitute personal in- 
teraction and judgment for quantita- 
tive performance indicators” and 
“more difficult to consistently inter- 
pret more complex performance indi- 
cators”.] 

What Bartik says is new is a range of 
performance indicators, from quite sim- 
ple, timely, easily quantifiable measures 
that are admittedly “rough proxies” for 
program goals, to “more sophisticated, 
accurate and complex approximations” 
of these goals. States, he says, have de- 
veloped a variety of performance mea- 
sures, and many have established incen- 
tive/sanction systems to encourage their 
use. 

The emphasis on a range of indica- 
tors is reinforced, according to Bartik, 
by the 1995 GAO study and a report by 
the American Public Welfare Associa- 
tion. States leading in the performance 
system movement have used measures 
of an array of outcomes, including ini- 
tial, intermediate, short-term (at termi- 
nation) and longer-term outcomes. They 
have also negotiated goals with local of- 
fices, and adjusted them based on un- 
employment levels and the composition 
of caseloads. Within different state ori- 
entations, meeting and exceeding stan- 
dards have various consequences — 
from no impact on funding or program 
operation, to more acceptable perfor- 
mance levels, to extremely strong re- 
wards and penalties. In this sense, 
states clearly have different objectives 
for their performance management sys- 
tems. It is therefore difficult to identify 
any one system as being more useful 
than another. 

The privatization of program opera- 
tion and service delivery is another sig- 
nificant variant, where private firms are 
paid based in whole or in part on their 
performance using a set of criteria. Bar- 
tik suggests that privatization combined 
with large incentives to meet certain 
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performance standards raises important 
issues of feasibility, fairness and afford- 
ability. 

Guidelines for Developing 
Performance Indicators 

Bartik concludes his monograph with 
suggested guidelines for developing in- 
dicators or measures for performance 
systems in welfare-to-work programs. 
They have significant implications for a 
host of human resource efforts: 

1. Agreement is needed on a few easy- 
to-understand performance measures 
that are frequently measured and 
which focus staff on the program’s in- 
tended goals. 

For example, the author of the report 
on the ET Choices program concluded 
that its apparent success was dependent 
on having a simple goal consistent with 
its mission. The same conclusion held 
for the JOBS program in Maryland, 
which focused on three simple goals: 
the number of participants, the number 
of participants leaving the program due 
to increased earnings, and the job reten- 
tion rate for those who obtained employ- 
ment. 

Simple systems, Bartik insists, have a 
more immediate effect in motivating 
staff than more complex ones. But these 
simplified systems are not without prob- 
lems. Simple goals, he says, may con- 
tain inconsistencies or actually conflict. 
This problem is illustrated by the five 
categories of JOBS goals: job placement, 
placement quality, reduction of welfare 
payments or getting off welfare, job and 
welfare reduction retention, and in- 
creased job skills. If all these goals are 
to be measured, a complex system is in- 
evitable, even though vigorously pursu- 
ing only a few goals may lead programs 
in the wrong direction. 

2. A performance indicator system 
does not need to be totally fair to be a 
useful motivator. 

Since it is impossible, in Bartik’s 
view, to measure perfectly or easily all 
the major social goals of a program, 
performance incentives and sanctions 
may not always reward or punish per- 
formers appropriately. The utility of a 
performance measure, he says, should 
be defined in terms not only of its accu- 
racy in representing the program’s goals 
but also its ability to resist managerial 
manipulation associated with meeting 
performance standards or targets. Stan- 
dards clearly need to “meet some mini- 



mum fairness criterion”. The perceived 
fairness of a performance management 
system may depend on whether the sys- 
tem affords flexibility regarding influ- 
ences not captured by quantitative ad- 
justments, such as reasons certain areas 
find it difficult to meet a particular stan- 
dard. Fairness is also related to the size 
of rewards and sanctions and how hard 
standards are to meet. 

3. “Creaming” may be necessary and 
good, if it is the right kind. 

Bartik says that making a selection 
among welfare clients in providing ser- 
vices sometimes supports the purpose 
of programs. For example, creaming that 
increases job development may have a 
positive value, since job development 
usually requires some level of selectivi- 
ty. One reason for the success of effec- 
tive job development, Bartik believes, is 
that employer information on job appli- 
cants and new hires is not very good, 
leading to costly employee turnover. 
Consequently, employers often hire the 
employees, friends or contacts of the 
employer, making it more difficult for 
those without a job development advo- 
cate to be hired. In addition, new hires 
obtained in this way are more subject to 
the prejudice of employers. Screening 
by programs to identify and market the 
more productive members of caseloads 
can lead to better goal achievement. 

Bartik’s qualification to this conclu- 
sion is that strong enforcement of laws 
against discrimination and careful mon- 
itoring of job developers’ attitudes and 
practices are necessary in order to pre- 
vent inappropriate screening. He agrees 
with Daniel Friedlander of MDRC that a 
“triage” approach to selecting program 
participants may increase goal achieve- 
ment, where caseloads are classified 
into subgroups based on the extent of 
their prior earnings and welfare history. 
In a 1988 study of employment and 
training programs for welfare clients, 
Friedlander found that there were posi- 
tive earnings impacts for the “mid-de- 
pendency tier” but none for subgroups 
in the “most dependent tier”. The least 
disadvantaged clients tended to increase 
their earnings without assistance from 
programs. The message was that the 
main goal of these programs (earnings 
gains) would be benefited by targeting 
welfare recipients with “average difficul- 
ty". 

The wrong kind of creaming clearly 
involves targeting clients whose out- 
comes will best serve the purpose of 
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meeting performance standards but may 
not represent the achievement of pro- 
gram goals. The targeting of certain cli- 
ents, Bartik suggests, must therefore be 
determined externally, rather than by 
staff whose behavior is judged against 
performance standards. The main basis 
for judging compliance with standards 
must be the entire caseload, Bartik says, 
or an entire segment of the caseload, as 
suggested by a 1995 U.S. Health and 
Human Services report to the Congress. 

4. Data on desired program outcomes 
for welfare clients should be adjusted 
through regression models so that the 
adjusted numbers are positively corre- 
lated with the true goals of the pro- 
gram. 

Daniel Friedlander and Jeffrey Zor- 
nitsky, of Abt Associates, examined the 
extent of the positive correlation be- 
tween adjusted outcome data and pro- 
gram goals. The study suggested that 
when individual outcomes are adjusted 
for the individual’s characteristics and 
the characteristics of the local economy, 
they are often significantly correlated 
with the goals of the program for that 
individual. This kind of adjustment is 
an incentive to avoid creaming. It also 
increases the perceived fairness of the 
performance system. And it is critical to 
replicating the program. 

Friedlander found that even very 
primitive adjustments to outcome data 
can yield significant positive correla- 
tions. Zornitsky found these kinds of 
adjustments for individuals to be 
significantly positively correlated with 
estimated earnings gains. Entered em- 
ployment rate, placement wage, em- 
ployment status and average weekly 
earnings at three and six months, and 
estimated reductions in average welfare 
payments were also correlated. 

5. Performance measures and stan- 
dards need to be as directly related to 
program goals as possible. 

In the Friedlander and Zornitsky 
studies, performance measures used to 
represent increased earnings and em- 
ployment were more strongly correlated 
with programs’ earnings goals than 
were welfare-reduction measures. Wel- 
fare reduction performance measures 
were more strongly correlated with the 
effect of welfare reductions on programs 
than were earnings and employment 
performance measures. Bartik’s conclu- 
sion is that performance measures for 
earnings goals are probably easier to de- 



velop than those for welfare reduction. 
And more adjustments to the indicator 
of welfare reduction are needed. A pro- 
gram can look successful simply be- 
cause many recipients who have in- 
creased their earnings are still 
dependent on the welfare system. 

6. It is more important to adjust for cli- 
ent characteristics than to extend the 
length of the measurement period. 

Extending measurement points be- 
yond the thirteen-week follow-up period 
helps, Bartik suggests, but it does not 
seem to make the correlations between 
performance indicators and the true 
achievement of program goals any 
stronger. Bartik recommends that in ad- 
justing performance indicators for the 
unique characteristics of each welfare 
office, researchers should consider these 
factors: 1 ) the prior earnings history, 
prior welfare history, race, marital 
status and gender of household head, 
education and age of the client, and 
2) measures that reflect both wages 
and job availability. 

7. Using performance systems to im- 
prove the achievement of program 
goals. 

A major objective of performance 
systems is the development of indica- 
tors that can be used in the short-run as 
indicators of long-term success, using 
the short run measures for program 
management purposes. Using long-term 
measures as a basis for incentives and 
sanctions would be ineffective for man- 
agers because of the delay in determin- 
ing compliance. 

Also, the use of such measures, such 
as more education, placement in better 
jobs, or targeting chronic welfare recipi- 
ents is not always supported by the 
data. Some programs emphasizing edu- 
cational interventions, for example, are 
successful and some not. The Minority 
Female Single Parent program in San 
Jose, and the New Chance program, are 
examples of programs emphasizing edu- 
cation and skill development that have 
been judged less than successful. ' 

It is difficult, Bartik says, to deter- 
mine from the data whether individuals 
placed in better jobs do better later be- 
cause of the placement, or whether in- 
dividuals with better skills get better 
jobs now and later. In the Friedlander 
study of the Baltimore program there 
was no evidence to indicate that welfare 
recipients obtained higher paying jobs 
than in other programs, even though the 
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programs had greater long-run success 
than other reforms. Bartik concludes 
that we do not yet know, and must do 
more research to determine, how to reli- 
ably increase the long-run earnings of 
individuals who would otherwise have 
been long-term welfare recipients. 

Performance Measurement 
Models 

Bartik’s concluding section suggests 
a number of performance models: 1) for 
state monitoring of local office perfor- 
mance, 2) for state analysis of those 
program approaches that appear to have 
the greatest long-term effectiveness, and 
3) for local office monitoring of staff 
and contractors. This monitoring hierar- 
chy involves all levels of a decentralized 
social program environment within a 
single but flexible performance mea- 
surement system. Clearly it requires a 
substantial change in the managerial 
culture of social programs. 



Editor’s Note 



Some of the studies referred to in 
the Bartik monograph are these: 
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1994. 
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ment Rates Under the Job Training Part- 
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Public Managers from the Massachu- 
setts Welfare, Training and Employment 
Program . Cambridge, MA: Harvard Uni- 
versity Press, 

Friedlander, D. Subgroup Impacts and Per- 
formance Indicators for Selected Welfare 
Employment Programs, New York, NY: 
MDRC, 1988. 

Friedlander, D. and G. Burtless. Five Years 
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to-Work Programs. New York, NY: Rus- 
sell Sage Foundation, 1994. 

Nightingale, D. et al. Evaluation of the Mas- 
sachusetts Employment and Training 
Program, Washington, D.C.:The Urban 
Institute, 1991. 
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New York, NY: MDRC, 1 994. 

Zornitsky, J. and M. Rubin. Establishing a 
Performance Management System for 
Targeted Welfare Programs. Washington, 
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Other W E. Upjohn Institute mono- 
graphs by Timothy J. Bartik that may be 
of interest: 

The Effects of Metropolitan Job Growth on 
the Size Distribution of Family Income, 
1991. 

Federal Policy Towards State and Local 
Economic Development in the 1990s, 
1993. 

The Effects of Local Labor Demand on Indi- 
vidual Labor Market Outcomes for Differ- 
ent Demographic Groups and the Poor, 
1993. 

Economic Development Strategies, 1995. 

Can Economic Development Programs Be 
Evaluated, 1995 (With R. Bingham) 



o ■ 



The Potential Effects 
of Performance 
Standards Systems 



with the assistance of 

Ernst W. Stromsdorfer 

Department of Economics 
Washington State University 



In this article we review an early 
draft of a significant book to be pub- 
lished soon by the W. E. Upjohn Insti- 
tute for Employment Research, Perfor- 
mance Standards in a Government 
Bureaucracy , edited by James J. Heck- 
man. A well known labor economist at 
the University of Chicago, Heckman has 
contributed important information in 
the draft manuscript to those who have 
watched fascinated as the performance 
measurement movement has unfolded, 
curious and anxious about its longer- 
term implications. 

Heckman is Director of the Center 
for the Evaluation of Social Programs in 
the Irving B. Harris School at the Uni- 
versity, and is Henry Schultz Distin- 
guished Service Professor in the Univer- 
sity of Chicago’s Department of 
Economics. The manuscript has the 
strong imprint of Heckman’s thinking 
and recent research by way of several 
chapters co-authored with colleagues at 
the Center, and through the general 
framework he developed for integrating 
the manuscript. 

The editor of Evaluation Forum and 
the guest reviewer, Ernst Stromsdorfer, 
would like to express our appreciation 
to Allan Hunt, W. E. Upjohn Institute, 
for permitting us to write a review arti- 
cle based on a draft copy of the manu- 
script. We also thank the authors of var- 
ious chapters for their useful review of 
our article and their suggestions for im- 
proving it. 

Why We Need to Know the Effects 
of Using Performance Standards 

The performance measurement 
movement has taken hold in all the 
postindustrial nations in the 1990s. 
There is a direct relationship between 
the growth of central governments over 
the past few decades, increasingly wor- 
risome central government deficits, the 
decentralization of social initiatives, and 
the need for central governments to 
compensate for the loss of control over 



social programs receiving central gov- 
ernment money by focusing account- 
ability efforts on their results — i.e., 
their performance. Performance stan- 
dards have become a major tool in this 
new effort to be accountable in the 
midst of substantial government re- 
structuring and downsizing. 

Little conclusive research has been 
conducted on the effect of performance 
standards on decisionmaking. This is 
partly due to the current political and 
bureaucratic popularity of the strategy. 
However, it is important that we give 
performance measurement attention pri- 
or to applying performance standards to 
increasing numbers of national pro- 
grams. The most critical reason the per- 
formance-oriented movement should be 
studied for its impact is the need to un- 
derstand whether, to what extent, and 
in what ways performance standards, 
reinforced by financial rewards and cor- 
rective sanctions, may redirect the atti- 
tudes, decisions and actions of those 
setting program policy and managing 
programs away from a program's in- 
tended goals, objectives, implementation 
strategy, and desired outcomes. We need 
to determine the value or detriment in 
any redirection that is occurring. 

There have been persistent worries 
about performance-driven program de- 
signs in terms of their potential for dis- 
torting or reformulating policies and 
procedures for selecting program partic- 
ipants — that is, redirecting the selec- 
tion process away from the program's 
selection objectives. In employment and 
training programs, the temptation to 
"cream” the eligible population of cli- 
ents for the higher-skilled individuals, 
whose performance is most likely to as- 
sist managers in meeting performance 
standards, has been a major concern of 
policymakers and evaluators. 

For example, a 1991 U.S. General Ac- 
counting Office report claimed that per- 
formance standards had resulted in seri- 
ous inequities, reducing the access of 
women and minorities to the program, 
to training within the program, and to 
the more productive training alterna- 
tives. The researchers found evidence 
that only the “most employable” disad- 
vantaged workers were being served. 
They attributed this result to the strong 
influence of incentive payments and the 
prestige accorded “high performance.” 

However, Heckman cites other pos- 
sible sources of program bias, such as 
management decisions favoring certain 
client characteristics that are unrelated 
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to a person’s suitability for training or a 
concern over equity issues, and the per- 
sonal choices of eligible clients them- 
selves. Anticipating that these criteria 
may be quite important, Heckman pro- 
poses that formal eligibility and mana- 
gerial acceptability should be treated as 
distinctly different participant selection 
criteria, whose effects need to be care- 
fully identified and analyzed. 

Assumptions Underlying 
Performance Measurement 
Systems 

Heckman’s explanation for the grow- 
ing interest in performance measure- 
ment is the belief that government is 
neither sufficiently efficient nor appro- 
priately sensitive to its customers’ 
needs and concerns, resulting in pres- 
sure for greater privatization and gov- 
ernment reform. The premise rationaliz- 
ing privatization, he says, is that market 
competition tends to lower costs and in- 
crease sensitivity to customers. Conse- 
quently, the argument goes, the seg- 
ment of government that remains 
unprivatized needs to borrow heavily 
from market concepts, such as market- 
oriented incentives that can push gov- 
ernment towards greater efficiency and 
responsiveness. 

Heckman views a system of perfor- 
mance standards, and a financial reward 
system supporting the achievement of 
these standards, as the most important 
aspects of this transplantation of private 
sector ideas into government activity. 
The goal of this version of government 
reform is to simulate private sector in- 
centives and competitiveness in a non- 
market environment, energizing govern- 
ment to be more cost-effective and 
satisfying to its customers. 

The Test Case 

Heckman and his colleagues focus 
their analyses on the behavioral impli- 
cations of performance standard sys- 
tems for the JTPA program, a reworked 
and expanded model of which has been 
proposed by the Clinton Administration. 
It is an appropriate model beyond its 
political saliency. The program’s legisla- 
tive design incorporates performance 
standards and incentives for meeting 
them — JTPA is the first national pro- 
gram to make these elements a require- 
ment. 

In addition to using a major ongoing 
national program as the book’s case 
study, the team of analysts benefited 



from the extensive data available on 
JTPA through the experimental National 
JTPA Study completed in 1995. These 
data included information for each of 
sixteen study sites concerning 1) the 
structure of incentives, 2) changes in 
these incentives over the period of the 
study, and 3) the nature of the decision- 
making process and its outcomes. 

The series of re-examinations of the 
experimental data which are discussed 
in the book represent a test of an impor- 
tant cause-effect relationship: do perfor- 
mance standards and incentive systems 
change a program's intended design — 
its intended participants, interventions , 
implementation strategy, and outcomes? 

The major research question for 
Heckman and his research group, across 
the series of studies, was how market- 
simulating systems affect the behavior 
of bureaucratic administrations and 
managers. More specifically, the search 
for answers to the following questions 
guided the analyses: 

■ How do performance standards oper- 
ate to exclude or include persons 
with different characteristics? 

■ Do performance standards affect the 
type of services offered? 

■ Do the outcome measures used in 
developing the performance stan- 
dards predict the long-term gains in 
participants’ earnings that are the 
primary goal of JTPA? 

■ How successful are performance 
standards in achieving their own 
stated goals? 

■ What general lessons can be learned 
from a study of the effects of varia- 
tion in the rules governing JTPA em- 
ployment and training centers? 

These questions did not emerge in a 
vacuum, even though the literature on 
the impact of performance standards 
has been meager. In addition to the 
GAO study, the 1988 SRI International 
study of the effects of performance 
standards on participants’ choice to en- 
ter the program, their choice of training 
options, and the cost of training sug- 
gested that an emphasis on perfor- 
mance standards decreased services to 
the “hard-to-serve.” A study of Tennes- 
see’s JTPA client population in 1993 re- 
vealed a disproportionate number of in- 
dividuals in the participant population 
who had a high probability for success- 
ful job placement, one of JTPA’s key 
performance standards. 



O ■ 

nr 



However, these studies did not bene- 
fit from detailed longitudinal micro-level 
information on stages in the selection 
and participation process. This informa- 
tion greatly assisted Heckman and his 
fellow evaluators in isolating the role 
played by bureaucratic decisionmaking. 
Furthermore, the experimental study 
contained information on over 20,000 
individuals, permitting an analysis of 
the effects of events in the recent em- 
ployment history of these individuals on 
their probability of being selected into 
JTPA. The National JTPA Study also 
provided data on th e JTPA- eligible non- 
participants in the geographical areas of 
the sixteen sites, and afforded rich data 
about variations in the applications of 
performance standards in these sites. 

MajorThemes 

Heckman’s chapter, “Lessons from 
the JTPA Incentive System,” which ends 
the manuscript, summarized the major 
insights emerging from the series of 
studies. These are the significant “les- 
sons” Heckman distilled from the analy- 
ses: 

Performance standards exerted a 
powerful influence on management 
and staff behavior. Although JTPA fi- 
nancial incentives are comparatively 
low and their benefits general, they 
have a strong influence. This is due to 
their value in increasing the profession- 
al stature and prestige of bureaucrats, 
and the leverage that awarded dollars 
create with the human services commu- 
nity in which administrators and man- 
agers wish to be credible players. 

Even small rewards are clearly signif- 
icant. JTPA agencies were making their 
placement, enrollment and termination 
decisions consistent with the need to 
meet performance standards. Manage- 
ment and staff practices demonstrated 
considerable innovation in manipulating 
various components and features of the 
program to boost measured perfor- 
mance. These practices included enroll- 
ing persons only after they obtained a 
job, choosing short-term training as a 
way to increase the probability of suffi- 
cient placements, holding less employ- 
able individuals in job clubs, and termi- 
nating those doing poorly in training 
slots at a point advantageous to meeting 
performance standards. Such practices 
are clearly rational and efficient in 
meeting performance standards, but 
contribute little in raising participants’ 
earnings or increasing service equity. 

The short-term performance mea- 
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sures used in setting standards were 
weakly (and often negatively) related 
to longer-term effects on employment 
and earnings. Heckman sees JTPA’s in- 
tended goals as long-term effects, not 
the short-term requirements in perfor- 
mance standards. Neither the original 
set of standards, to be applied at a par- 
ticipant’s termination from the program, 
nor the new set mandated in the JTPA 
amendments, to be applied 13 weeks 
following termination, were highly cor- 
related with JTPA’s intended impacts. 
Consequently, the influence perfor- 
mance standards clearly had on mana- 
gerial behavior did not appear also to 
promote long-term wage gains or posi- 
tive employment outcomes for partici- 
pants. 

JTPA performance standards did 
not result in any significant “cream- 
ing.” Although previous research sug- 
gested the possibility of creaming, for 
most demographic groups the earnings 
effects were uniform over a range of 
skill levels. Only in the lowest skill cate- 
gory was there evidence of smaller im- 
pacts. There was a trade-off between 
serving the most disadvantaged and us- 
ing resources most efficiently, but only 
a very modest one given JTPA’s modest 
impacts. Administrators’ scope of au- 
tonomy and discretion played a role in 
explaining differences in client partici- 
pation rates but not the predominant 
one. Information about the program and 
its service options, and the personal 
choices of clients played a more impor- 
tant part in explaining the disparities in 
participation rates for different groups. 
The more employable and more easily 
placed applicants were more likely to be 
encouraged to enroll, but these were the 
individuals most likely to be more 
aware of the program and interested in 
applying. 

The Focus of the Manuscript 

The manuscript is organized interest- 
ingly, with the clear objective of provid- 
ing the reader with a detailed under- 
standing of the important steps in the 
JTPA process from client application 
through service delivery to program out- 
comes. This progression serves as a 
framework for reviewing the rigorously 
conceived and carefully conducted stud- 
ies of how the JTPA performance stan- 
dards system is likely to serve as an un- 
measured but critical program 
intervention. 

Jeffery Smith, a faculty member at 



the University of Western Ontario, Can- 
ada, and an associate of the Center, 
studies four of the National JTPA Study 
sites to 1) document the extent to 
which diverse demographic groups dif- 
fered in their proportion in the partici- 
pant population at various steps in the 
selection process, 2) determine the de- 
mographic characteristics of the eligible 
JTPA population in the four sites who 
had heard of the program, were aware 
they were eligible, who were applicants 
previously, or who had been partici- 
pants at some prior time, and 3) provide 
an empirical basis for designing econo- 
metric selection estimators for a subse- 
quent study of the selection process. 

Pascal Courty and Gerald Marschke 
of the University of Chicago Economics 
Department and the Center, study the 
organization of the “JTPA bureaucracy,” 
in particular its unique governance 
structure. They examine how the JTPA 
incentive contracts associated with per- 
formance standards are developed and 
operate, how the authority for writing 
and implementing these contracts is dis- 
tributed across various political bodies, 
and how JTPA operatives exercise sub- 
stantial discretion in making JTPA re- 
source allocation decisions, 

Heckman and Smith then look at 
“creaming” in JTPA, studying the rela- 
tionship between well-specified defini- 
tions of “creaming” and JTPA’s compet- 
ing goals. The purpose is to determine 
the potential importance of creaming. 
They identify three major concerns: 

1) efficiency — in that it is believed that 
the least advantaged eligibles are most 
likely to benefit from JTPA and cream- 
ing may reduce their participation, 

2) equity — in that the program seeks 
greater equity in resource distribution, 
and 3) fairness — in that the hard-to- 
serve may be under-represented in the 
participant group due to discriminatory 
practices. 

Heckman alone examines the issue 
of whether short-run performance 
goals, such as those used in the JTPA 
performance standards system, predict 
long-run impacts of the kind desired in 
the JTPA legislation. If these short-term 
outcomes are not positively correlated 
with longer-term goals, the assumption 
is tenable that these standards have 
guided bureaucratic behavior away from 
legislative intent. 

Courty and Marschke follow Heck- 
man’s analysis with a study of actions 
by JTPA bureaucrats which may be mo- 



tivated primarily by performance 
awards. They focus their attention on 
the performance measure receiving the 
greatest emphasis, the employment rate 
at termination. 

In all cases, the analysts look for evi- 
dence of effects that could be attributed 
to the JTPA performance/incentives sys- 
tem, separate from those associated 
with JTPA’s formal service interven- 
tions. 

The JTPA Selection, Assessment 
and Assignment Process 

Smith’s work concentrates on the 
four sites within the sixteen sites stud- 
ied in the National JTPA Study, where 
data were collected on JTPA-eligible in- 
dividuals not participating in the pro- 
gram. These data made it possible to ex- 
amine more fully the “front-end” of the 
service delivery system. 

Smith compares the eligible popula- 
tion at the four sites with the National 
Study’s experimental groups, the latter 
consisting of individuals who had com- 
pleted the application-through-assess- 
ment process. Additional information 
was collected on all program applicants 
at one of the sites and compared with 
the experimental group, to establish 
what kinds of individuals with what 
sorts of characteristics were most likely 
to drop out in the application-assess- 
ment process. 

The main questions of interest in the 
cross-site comparison were: 

■ What differences exist in the demo- 
graphic characteristics of applicants 
and enrollees? 

■ What are the differences regarding 
the probability that enrollees will ac- 
tually receive training (i.e., with the 
exception of those randomly selected 
into the study’s control group)? 

■ What are the differences in the types 
of services enrollees receive? 

The Selection Process 

In comparing the experimental sam- 
ple with a sample of JTPA eligibles in 
the four sites, Smith found a substantial 
amount of selection based on demo- 
graphic characteristics at all stages in 
the process of application through the 
receipt of services. Significant demo- 
graphic differences existed between the 
eligible population and the group com- 
pleting this process in terms of aware- 
ness of JTPA and the probability of re- 
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ceiving particular kinds of services once 
in the program. Eligible out-of-school 
youth were more likely than eligible 
adults to be conscious of the program 
and their own eligibility, and to have 
applied or been participants in the past. 
These differences may have been relat- 
ed, Smith suggested, to patterns in the 
cost/benefit trade-off associated with 
participation, to local decisions, and/or 
to local economic environments. 

Smith’s major findings were these: 

■ Women constituted a sizeable major- 
ity (65%) of the eligible population 
in all four sites, and the proportion 
was roughly the same across sites. 

■ There were substantial differences 
across sites in the racial/ethnic com- 
position of the eligible sample (one 
was 80% white; another 80% His- 
panic Americans). 

■ Over 10% of the eligible sample at 
the four sites had previously applied 
to JTPA and over half of these appli- 
cants reported having received JTPA 
services in the past. 

■ African Americans were more likely 
to have applied previously than 
whites or Hispanic Americans. Out- 
of-school youth were more likely 
than adults to have applied previous- 
ly. 

■ Eligible African Americans who had 
applied previously were more likely 
to report the receipt of JTPA services 
than were previously-applying 
whites or Hispanic Americans. 

■ Barely more than 40% of the eligible 
sample had heard of JTPA, and of 
these just under half realized they 
were eligible. In total, only 18.4% of 
eligible individuals had heard of the 
program and understood they were 
likely eligible. 

■ African Americans were more likely 
than whites to have heard of the pro- 
gram; whites more likely than His- 
panic Americans. Among out-of- 
school youths, African Americans 
were more likely to be familiar with 
the program than whites or Hispanic 
Americans. 

■ The low rate of program awareness 
among Hispanic Americans was at- 
tributed in part to the language barri- 
er. 

In comparing the experimental sample 
with the sample of eligibles, Smith finds 



a number of demographic differences 
that may have been related to patterns 
in the cost/benefit trade-off for partici- 
pants, to local decisions, and to local 
economic conditions. An analysis of the 
one site providing systematic data for 
applicants during the period of the Na- 
tional Study showed that men, local mi- 
nority groups, and out-of-school youth 
were more likely to complete the appli- 
cation and assessment process than ei- 
ther African American or Hispanic 
American applicants, and that adults 
were more likely to do so than out-of- 
school youth. 

Assignment to JTPA 
Treatments 

Following assessment, and prior to 
being randomly assigned to control vs. 
experimental groups, JTPA participants 
in the National JTPA Study were as- 
signed to one of three major service in- 
terventions, presumably based on the 
assessment information. These interven- 
tions were: 1) classroom training in oc- 
cupational skills (CT-OS), 2) on-the-job 
training (OJT), and 3) “other services” 
(mainly job readiness preparation, job 
search assistance, try-out employment 
and/or various non-training services). 

In comparing assignments across the 
four study sites, the key findings were 
these: 

■ Sites differed in the proportion of cli- 
ents in the experimental sample as- 
signed to each of the three service 
interventions. Some sites empha- 
sized OJT, for example. 

■ The four sites had a higher rate of 
assignment to CT-OS and a lower rate 
to OJT and “other services” than did 
the remaining twelve experimental 
sites in the National Study. 

■ Whites had a higher probability of 
assignment to OJT than African 
Americans or Hispanic Americans. 

■ Out-of-school youth were more likely 
than adults to be assigned to CT-OS. 

The Receipt of Services 

Assignments did not always predict 
the services individuals actually re- 
ceived. In this respect, Smith’s analysis 
is quite enlightening. Only 58% of the 
individuals completing the application 
through assessment process at the four 
study sites actually received one of the 
three JTPA service interventions. Most 
of these individuals received only one of 



the three types of services, whereas in 
the other twelve experimental sites 64% 
received one of these service interven- 
tions. Over 73% of those assigned to 
CT-OS in the four study sites actually re- 
ceived classroom training, whereas just 
over 51% of those assigned to OJT in 
the other experimental sites received 
this intervention. 

In the four sites, African Americans 
had a lower probability of receiving any 
of the three service alternatives. The 
differential was larger for African Amer- 
ican youth assigned to OJT. Both Afri- 
can American and white adults were 
substantially less likely to receive ser- 
vices than at the other experimental 
sites. Out-of-school youth in the four 
sites were more likely to receive servic- 
es than adults, but less likely to receive 
them than in the other experimental 
sites. 

Routinely collected JTPA administra- 
tive data classifies services into class- 
room training, OJT, job search assis- 
tance, basic education, work exper- 
ience, and non-training services. The 
management information system reports 
who received these services. Based on 
these data, Smith finds that 1) women 
were more likely than men to receive 
classroom training and men more likely 
than women to receive OJT; 2) out-of- 
school youth were more likely than 
adults to receive non-training services, 
with adults more likely than youth to 
receive job search assistance; 3) the 
study sites provided less basic educa- 
tion, work experience and non-training 
services than other experimental sites, 
and more job search assistance; and 4) 
Hispanic Americans had very different 
service receipt patterns than those in 
the other sites, being more likely in the 
four study sites to receive non-training 
services — although this result was 
skewed by the site with the largest 
number of Hispanic American partici- 
pants. 

The Incentive System 

JTPA is described by Courty and 
Marschke as embodying “the spirit of 
the New Federalism in the Reagan 
era” — that is, it has a highly decentral- 
ized program design. The major out- 
comes are to be increased employment 
and earnings, and a reduction in welfare 
dependency. A distinctive feature of the 
program is an incentive system that re- 
wards service deliverers that perform 
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well based on a set of “performance 
standards.” JTPA is, in fact, the first 
large-scale national program in the U.S. 
that uses financially-based performance 
measures and standards, a feature later 
incorporated in the JOBS program for 
welfare clients and proposed for a num- 
ber of other national initiatives. 

States must set aside 5% of their 
JTPA appropriation to fund this incen- 
tive and sanction system. Congress, the 
U.S. Department of Labor and the states 
all play a role in defining performance 
standards and incentive/sanction poli- 
cies. This involves determining the 
quantitative measures that will repre- 
sent key outcome variables, setting nu- 
merical standards, and developing a 
model for adjusting standards that ac- 
counts for influences considered beyond 
the control of the service providers. 

In 1984 the Department of Labor es- 
tablished seven performance mea- 
sures — for youth, adults and welfare 
recipients. In 1988 the Department in- 
troduced new measures that included 
"post-program” outcomes, to encourage 
the provision of more intensive training. 
In 1992 the cost measures in perfor- 
mance standards were eliminated and 
measures to encourage providers to 
serve the “hard to place” were added, 
based on a U.S. General Accounting Of- 
fice study suggesting that JTPA manag- 
ers were discriminating against certain 
kinds of eligible clients in the selection 
process. 

Currently there are four core perfor- 
mance measures for adults: two post- 
program employment measures and two 
average earnings measures; one set for 
all adults and one set for welfare cli- 
ents. Some states have added their own 
measures, such as expenditure mea- 
sures, “system-building” measures, and 
measures of employment retention. 

A regression model is used to esti- 
mate the importance of factors outside 
the providers’ control, such as the de- 
mographic and economic characteristics 
of the providers’ geographical area. 
There are several important aspects to 
the regression model. It must involve 
measures of the independent explanato- 
ry variables and the dependent variable. 
It must identify which variables have 
the most significance and why. Then 
problems must be anticipated with re- 
spect to the assumptions underlying the 
model or constraining its generality. 

Such problems must be given atten- 
tion. For example, performance stan- 
di 




dards vary with the enrollment and ter- 
mination decisions of training providers 
within a particular geographical region; 
therefore, these providers can manipu- 
late the standards. Also, the model may 
not hold training providers harmless for 
serving individuals with different char- 
acteristics, because it is based on pro- 
vider-level rather than individual-level 
data. In addition, adjustment factors are 
calculated using the participant popula- 
tion rather than the eligible population. 

These features aside, Courty’s and 
Marschke’s analysis indicates that feder- 
al guidelines for computing the differ- 
ence between actual performance and 
desired performance (as measured by 
standards) gave states and service pro- 
viders substantial flexibility in interpret- 
ing the guidelines. This flexibility gave 
them leeway to manipulate the perfor- 
mance standards system as well as to 
meet or exceed the standards. Also, in- 
centive contracts specifying the condi- 
tions under which financial awards 
were to be given varied considerably 
across states and years, suggesting that 
states used their discretion in designing 
these contracts, often rewarding service 
providers for enrolling certain target 
groups of eligibles. 

The researchers characterize their 
study as offering “compelling evidence” 
that JTPA organizations “game” the in- 
centive system. The analysis provided, 
they say, the first hard data on how dif- 
ficult it is to measure performance ap- 
propriately and how lacking in “robust- 
ness” different kinds of incentive 
systems are in avoiding “gaming” be- 
havior. The study showed that the in- 
centive system was likely encouraging 
job training organizations to “ waste re- 
sources in counterproductive actions in 
order to maximize their performance 
scores. ” That is, such systems may have 
been increasing the budgets of these or- 
ganizations based entirely on those or- 
ganizations’ successful gaming strate- 
gies. 

Based on this central insight from 
the study, the authors suggest that the 
JTPA incentive system is implicated in 
allocating resources disproportionately to 
organizations that may have been ineffi- 
cient or ineffective vis-a-vis JTPA’s long- 
er-term goals. In this sense, the re- 
searchers see the potential for the 
incentive system to become “a moral 
hazard,” maximizing narrow measures 
of performance at the expense of job 
training organizations’ intended objec- 
tives. 



Their conclusions are that 1) JTPA 
organizations tended to terminate em- 
ployed enrollees at the end of their 
training spells while delaying the termi- 
nation of unemployed enrollees, and 
2) these organizations terminated a 
large fraction of participants at the end 
of the accounting year. They attribute 
the latter finding to the annual periodic 
nature of performance measurement 
which was accentuated by performance 
thresholds. They go further, suggesting 
that the phenomenon of maximizing the 
incentive award is true of any explicit 
incentive system incorporating perfor- 
mance measures that are not well- 
aligned with the intended goals of the 
organizations involved. 

Using Performance Standards to 
Motivate Program Organizations 

Courty and Marschke test whether 
JTPA bureaucrats respond to changes in 
performance incentives. Using descrip- 
tions of incentive policies for each of 
the sixteen National Study sites, admin- 
istrative data on inputs, outcomes and 
enrollment/training decisions, and a 
freshly-designed survey of JTPA manag- 
ers and staff, the researchers study the 
extent to which the relatively weak fi- 
nancial link between performance and 
incentive awards was given serious con- 
sideration by JTPA managers. The au- 
thors give attention to the nonfinancial 
reasons bureaucrats may view incentive 
policies useful in managing their pro- 
grams, such as increased career pres- 
tige, higher salaries or promotions, po- 
litical gain, protection from criticism, 
and general credibility. 

What the researchers find is quite in- 
teresting. There was a strong pattern 
across fourteen of the sixteen National 
Study sites of terminating a large pro- 
portion of JTPA participants in the last 
month of the contract year, which the 
researchers maintain was consistent 
with "an award-maximization model of 
job training center behavior.” Based on 
“participant inventory control strate- 
gies,” the researchers explain this pat- 
tern in terms of the high marginal value 
in meeting performance standards 
which was achieved by waiting until the 
end of the contract year to terminate 
unemployed participants. 

The researchers find that the nonem- 
ployed were driving the end of the year 
termination pattern. End-of-year ter- 
minees showed significantly lower out- 
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comes on all performance measures for 
all years, and enrollees terminating at 
the end of the contract year had longer 
enrollment spells than enrollees termi- 
nating in other months. This is consis- 
tent, the researchers conclude, with 
maximizing the likelihood of an incen- 
tive award by sorting individuals into 
the year in which their marginal value 
was highest. 

Although the authors suggest that 
further research is needed, they deemed 
definitive their tests of the validity of 
their hypothesis that incentives change 
bureaucratic behavior and lead to a sig- 
nificant change in measured outcomes. 
They claim that bureaucratic actors do 
respond to incentives by managing their 
inventories of trainees in the interest of 
obtaining an award, and that this be- 
havior in fact does lead to different (re- 
ported) program outcomes. 

“Cream-Skimming” in JTPA 

Analyses conducted on “cream skim- 
ming” by Heckman and Smith, using 
data from the National JTPA Study, re- 
veal a surprising lack of positive corre- 
lation between the use of performance 
standards and the tendency in JTPA ser- 
vice delivery systems to select the more 
skilled and employable individuals. 
Making the various definitions of 
“cream-skimming” clearer, .they contend 
that evidence concerning the relation- 
ship between outcome levels and pro- 
gram gains is required if the efficiency 
of the cost/benefit tradeoff associated 
with skimming is to be established. 

Such evidence is necessary, they say, for 
judging the efficiency costs involved in 
concentrating program services on the 
least disadvantaged. 

The evidence produced by the re- 
searchers suggests that there is little effi- 
ciency loss for adults and for female 
youth associated with skimming. The re- 
lationship between income levels and 
the benefits of program participation is 
generally weak, except at the low end of 
the skill distribution. For these groups, 
targeting services to those at the middle ' 
of the skill distribution resulted in mini- 
mal loss in efficiency. For male youth, 
the effect of JTPA was generally nega- 
tive, but least so for the least disadvan- 
taged. 

Predicting Long-Term Effects 
from Short-Run Performance 
Measures 

In studying the consistency between 
short-term performance standards and 



longer-term JTPA goals, Heckman re- 
minds us that the formal purpose of per- 
formance management systems is to in- 
crease the extent to which programs 
meet their intended long-term goals . In 
JTPA, the primary long-term goal is an 
increase in employment and earnings. 
Direct measures of employment and 
earnings effects are expensive and diffi- 
cult to obtain. Consequently the JTPA 
performance management system uses 
alternative (proxy) measures consisting 
of reported levels of employment and 
earnings at program termination and 
shortly following termination. Heckman 
points out that if these alternative mea- 
sures are to be trusted as valid represen- 
tations of longer-term goals, short-term 
earnings gains must be shown to have a 
positive relationship with earnings gains 
beyond termination and the brief re- 
quired follow-up period. Otherwise the 
performance management system has 
not achieved its purpose. 

Using data from the National JTPA 
Study, Heckman finds that the short-run 
measures for earnings and employment 
used in the JTPA performance standards 
system are only weakly related to long- 
term measures. In many cases, there was 
a negative relationship between the 
short-run proxies and longer-term earn- 
ings effects. More specifically, in mea- 
suring earnings and employment im- 
pacts over several time intervals, 
Heckman finds not only that short-run 
performance measures are only weakly 
related to long-run gains, but that this 
relationship worsened as earnings were 
measured further and further from pro- 
gram termination. 

Also, performance measures were at 
greater odds with long-run earnings 
gains over the thirty months following 
random assignment, in the National 
Study sites, than over the first eighteen 
months. In addition, the relationship 
between some of the performance mea- 
sures and the impacts reported in the 
Study were weakly negative. For adults, 
the correlation between short and long- 
run measures were low and often nega- 
tive. Higher short-run measures of em- 
ployment for both men and women 
were consistently negatively related to 
estimated impacts for employment over 
the longer term. 

To illustrate the problem involved, 
Heckman cites the use of test scores in 
public education to motivate school sys- 
tems to improve scores. The problem is 
that test score gains may be due to 
teachers’ having learned to “teach the 



test” rather than reflecting real gains in 
long-run student achievement. 

Heckman’s conclusion from his anal- 
yses is that the JTPA performance man- 
agement system does not achieve its pur- 
pose to increase the earnings and 
employment of JTPA participants. He 
believes this evidence questions the 
utility of the current JTPA performance 
management system, with implications 
for the widespread use of such systems 
in other national programs, and in 
states where performance management 
has been gaining prestige. 

In this focal chapter of the manu- 
script, Heckman describes a general 
perspective on effective performance 
standards systems, suggesting three es- 
sential features: 

■ The goals of such a system, and the 
performance measures used, must be 
clearly defined. 

■ Program staff must be motivated to 
attain the short-run performance tar- 
gets, based on a reward/sanction 
strategy that is clearly understood 
and does not divert staff from the 
stated longer-term goals of the pro- 
gram. 

■ Short-run targets must predict to de- 
sired long-term effects — otherwise 
motivating staff to meet them will 
not increase the program’s achieve- 
ment of critical goals. 

Some Comments on the 
Manuscript 

The upcoming book will constitute a 
landmark contribution to the fledgling 
knowledge base that informs managers 
and evaluators about the new subfield 
of “performance measurement.” A ma- 
jor question about the book remains, 
however: "have the quantitative analy- 
ses provided accurate and sufficient evi- 
dence that a causal relationship exists 
between the use of performance stan- 
dards and the program redirections they 
identify?” The researchers’ inferences 
about cause-effect are based largely on 
statistical analyses and predictive mod- 
eling. These inferences did not benefit 
from a combination of process and out- 
come evaluation methods, which could 
have included in-depth interviews and 
focus groups with the bureaucrats about 
whose behavior the authors generalize. 

Some experts may feel that the ab- 
sence of more qualitative information, 
and its integration with the results of 
econometric analyses, raise the question 
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of the extent to which the research team 
controlled for less tangible influences 
that could explain the documented redi- 
rection of JTPA’s longer-term legislative 
goals. It is conceivable that the team 
has attributed too much importance to 
the performance standards system due 
to too-narrow a conceptual view of the 
phenomenon they sought to under- 
stand. 

On the other hand, it is important to 
recognize that objective, systematic data 
on bureaucratic performance across a 
number of experimental sites is used in 
this series of analyses to model actual 
behavior in response to the performance 
system. The statistical models and their 
assumptions are clearly stated. The be- 
havior that is modeled is verifiable — it 
is not an artistic mosaic. Hypotheses are 
formulated clearly. They are tested ac- 
cording to standard statistical theory. 
They can be accepted or rejected with 
known degrees of probability. Further- 
more, the analysts’ work can be repli- 
cated by others to see if they have told 
the whole story, or to see if other inter- 
pretations of the same data are possible. 
This is not characteristic of most quali- 
tative studies, as valuable as they too 
can be. 

But the research team does not lack 
qualitative information on the role of 
administrative and service delivery deci- 
sionmaking, which informs the other 
findings. One of the most interesting 
chapters is Carolyn Heinrich’s case 
study of a large metropolitan area in Illi- 
nois. The focus is not only on the rela- 
tionships between performance stan- 
dards and other aspects of JTPA 
program administration and service de- 
livery but also on the implications of 
this relationship for program outcomes 
and impacts. The supporting data set 
includes all management information 
system data for 1984 through 1994, in- 
formation on contracts between the Ser- 
vice Delivery Area and its service pro- 
viders through June 1993, detailed 
information on individual participants, 
participant-observation data, and inter- 
views with administrators, service deliv- 
ery managers and program intake staff. 



Heinrich was interested in 1) dis- 
cerning the structure and functions in- 
volved in the selection of participants 
and their assignment to services — par- 
ticularly how and to what extent perfor- 
mance standards influenced these deci- 
sions, and 2) understanding how the 
SDA contracted with local service pro- 
viders and evaluated their performance 
vis-a-vis performance goals. What she 
found in this intensive study was quite 
enlightening: 

■ Performance standards tended to en- 
courage the selection of those partici- 
pants most likely to have good post- 
program outcomes even in the 
absence of services. 

■ Creaming had negative implications 
for the achievement of JTPA’s equity 
goals, and the maximization of earn- 
ings gains. 

■ The influence of performance stan- 
dards was exerted directly through 
participant selection and service as- 
signment processes involved in ter- 
mination-based contract provisions 
between the SDA and its service pro- 
viders. 

Heinrich used a simulation based on 
actual decision points in the participant 
selection and assignment process. The 
model consisted of four parts: 1) the se- 
lection of participants from a pool of 
applicants, 2) the assignment of select- 
ed participants to training services, 

3) the consideration of alternative per- 
formance standard and cost constraints 
operating to influence the previous deci- 
sions, and 4) an analysis of selection 
decisions by intake staff to explore the 
influence of applicant characteristics 
and external factors on these decisions. 
She then tested several propositions 
concerning relationships among these 
variables. In doing so, several key re- 
search questions were of interest: 

■ Is there a particular set of applicant 
characteristics that influences intake 
staff decisions? 

■ Is there general agreement across 
staff about what characteristics are 
most important in selecting, and are 
selection decisions similar across dif- 



ferent staff in terms of weighting and 

judging such characteristics? 

■ Were intake staff’s decisions across 

the applicant population consistent 

over time? 

Heinrich used logistic regression 
analysis to answer these questions with 
both simulated and actual data. In the 
simulation models, basic skills deficien- 
cies and limited work histories emerged 
as key selection factors. But an analysis 
of actual participant selections suggest- 
ed that staff selection decisions were in- 
fluenced heavily by staff perceptions of 
applicants’ probability of placement, 
which supported the tendency toward 
creaming. The kinds of training services 
and their availability frequently influ- 
enced the selection process. Implicit in 
this influence was the role of funding 
constraints and performance standards, 
which tended to direct attention to fill- 
ing less expensive, shorter-term training 
slots, a tendency inconsistent with rais- 
ing measured performance, increasing 
earnings impacts, and improving the 
program’s cost-effectiveness. 

Heinrich’s analysis also showed that 
years of schooling completed was posi- 
tively related to assignment to job 
search assistance. Given funding cut- 
backs and the relatively low cost of job 
search assistance, staff naturally tended 
to select more individuals with higher 
education levels, whereas those with 
basic skills deficiencies were significant- 
ly more likely to be assigned to remedi- 
al education. As slots for this more ex- 
pensive training option dwindled, fewer 
low-skill applicants were selected for 
program participation. 

Overall, the findings supported the 
existence of direct and indirect cream- 
ing. Heinrich concluded that perfor- 
mance standards were effective man- 
agement tools, but if not carefully 
aligned with intended program goals 
could lead to unintended outcomes. She 
recommended a redesign of the perfor- 
mance standards system, away from 
one oriented to placements and toward 
one more dedicated to long-term em- 
ployment retention and economic self- 
sufficiency. 
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Editor’s Note 



The following publications are key 
resources in understanding the use of 

performance standards systems in em- 
ployment and training: 

Anderson, K., R. Burkhauser, and J. Ray- 
mond, “The Effect of Creaming on Place- 
ment Rates Under the Job Training Part- 
nership Act," in Industrial and Labor 
Relations Review 46, 1993. 

Baker, G. R “Incentive Contracts and Perfor- 
mance Measurement," in Journal of Polit- 
ical Economy, June 1992. 

Barnow, B. “The Effect of Performance Stan- 
dards on State and Local Programs," in 
Evaluating Welfare Training Programs, 
edited by C. Manski and I. Garfinkel, 
Cambridge, MA: Harvard University 
Press, 1992. 

Brown, R., T. Gallagher, and M. Williams. Au- 
diting Performance in Government. New 
York: Wiley, 1982. 

Clements, N., J. Heckman, and J. Smith. 
Making the Most out of Social Experi- 
ments: Reducing the Intrinsic Uncertain- 
ty in Evidence from Randomized Trials 
with an Application to the National JTPA 
Experiment. Working Paper. Chicago: 
University of Chicago, 1995. 

Comptroller General of the U.S., U.S. Gen- 
eral Accounting Office. Standards for Au- 
dit of Governmental Organizations, Pro- 
grams, Activities, and Functions. 
Washington, D.C.: U. S. Government 
Printing Office, 1981. 

Dickinson, K., et al. Evaluation of the Effects 
of JTPA Performance Standards on Cli- 
ents, Services, and Costs . Research re- 
port. Washington, D.C.: National Com- 
mission for Employment Policy, 1988. 



Doolittle, F. and L. Traeger. Implementing the 
National JTPA Study New York: Manpow- 
er Demonstration Research Corporation, 
1990. 

Heckman, J. and J. Hotz. “Choosing Among 
Alternative Nonexperimental Methods for 
Estimating the Impact of Social Pro- 
grams: The Case of Manpower Training,” 
in Journal of the American Statistical As- 
sociation 84, 1989. 

Heckman, J. “Randomization and Social 
Program Evaluation," in Evaluating Wel- 
fare and Training Programs edited by C. 
Manski and I. Garfinkel. Cambridge, MA: 
Harvard University Press, 1993. 

Heckman, J., R. Roselius and J. Smith. “U.S. 
Education and Training Policy: A Re-eval- 
uation of the Underlying Assumptions 
behind the ‘New Consensus’," in Labor 
Markets, Employment Policy, and Job 
Creation edited by L. Solomon and A. 
Levenson. Boulder, CO: Westview Press, 
1994. 

Orr, L., H. Bloom et al. The National JTPA 
Study: Impacts, Benefits, and Costs of 
Title HA, Bethesda, MD: Abt Associates, 
1994. 

Romero, F. and J. Gonzales. Falling through 
the Cracks : Hispanic Underrepresenta- 
tion in the Job Training Partnership Act. 
Washington, D.C. National Council of La 
Raza, 1989. 

Schwandt, T. and E. Halpern. Linking Audit- 
ing and Meta-Evaluation. Newbury Park, 
CA: Sage, 1988. 

U.S. General Accounting Office, Job Training 
Partnership Act: Racial and Gender Dis- 
parities in Services. Report to the Com- 
mittee on Government Operations, 

House of Representatives, 1991. 

Please see also articles on perfor- 
mance issues in Issue #11 of Evalua- 
tion Forum. 
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Editorial Commentary 



Most of the authors of the articles, 
papers, monographs and books re- 
viewed in the series on performance 
measurement systems have made a 
compelling case for a results-oriented 
approach for redefining government ac- 
countability — an approach that reflects 
earlier public management concepts 
and borrows from the private sector. 

Some of the articles make an equal- 
ly dramatic case for studying the impact 
of using performance measures on the 
achievement of the intended goals of 
social initiatives. Clearly there is a 
down side to the use of performance 
systems, particularly in the absence of 
evaluation research. The inappropriate 
manipulation of programs in order to 
obtain incentives associated with per- 
formance standards systems is an ex- 
ample. 

Although some concern is being ex- 
pressed, internationally, about the po- 
tential negative effects of performance 
systems, the U.S. has been in the fore- 
front of studying such effects, as wit- 
nessed by the Upjohn Institute book 
edited by Jim Heckman. The recom- 
mendations flowing from these studies 
should serve as a caution to policymak- 
ers, agency administrators and pro- 
gram operators. They should suggest 
new ways to prevent a displacement of 
policy and program goals, if we are de- 
termined to participate in the perfor- 
mance measurement movement. 







Alternative Evaluation Approaches 



Editorial Introduction 



Some experts on the evaluation of 
youth programs have concluded that 
evaluators, and those authorizing re- 
search, need to consider a wider range 
of methodologies in studying the imple- 
mentation and outcomes of these pro- 
grams. The three articles in this section 
provide ideas about evaluating net im- 
pact when random assignment is not 
feasible; using a mix of different meth- 
ods; and developing a model for under- 
standing and judging complex interre- 
lated programs. 



Testing the Use of 
Program Applicants 
in Studying the Net 
Effects of Social 
Programs 

A guest review by 

Ernst W. Stromsdorfer 

Department of Economics 
Washington State University 



This review article comments on a 
recent publication from the W.E. Upjohn 
Institute for Employment Research writ- 
ten by Stephen H. Bell, Larry L. Orr, 
John D. Blomquist and Glen G. Cain ti- 
tled Program Applicants as a Compari- 
son Group in Evaluating Training Pro- 
grams. This study attempts to extend 
nonexperimental evaluation methodolo- 
gy by developing statistically unbiased 
estimates of the net economic impact of 
governmentally-subsidized training pro- 
grams — that is, to estimate net impact 
when there is no random assignment to 
treatment and control groups. In such 
nonexperimental situations, training 
programs may operate by means of a 
demonstration set up specifically to as- 
sess how a particular mix of treatments 
affects a specific target group or groups, 
or the programs being assessed may be 
operational programs. 

Concern over selection bias has ex- 
isted for over two decades, but thus far 
a solution to the selection bias problem, 
like the Holy Grail, has eluded discov- 
ery. The present study is no exception, 
though it is a very useful piece of work 
and should be mandatory reading for 
anyone who intends to understand and 
perform social program evaluation. 

The fundamental conceptual issue in 
evaluating a training program or any 
other social program is the issue of pro- 
viding statistically unbiased estimates of 
outcomes or benefits of the program 
treatment. Of course, since foregone 
earnings are often involved in the cost- 
benefit analysis of a social program — 
such as a training intervention — it is 
also necessary to provide unbiased esti- 




mates of program costs as well. A true 
experimental design, with random as- 
signment to treatment and control 
groups plus no nonrandom attrition or 
crossovers, will guarantee an unbiased 
estimate of program net benefits and 
any foregone earnings, should they ap- 
ply. 

However, even the best managed ex- 
periments will suffer from attrition and 
possibly crossover behavior. Conse- 
quently the problem faced by the au- 
thors of the Upjohn book must be ad- 
dressed even by those who design and 
operate experiments as well as by those 
who assess ongoing programs or dem- 
onstrations. The nonrandom selection 
into a program or attrition from a pro- 
gram are problems because the selec- 
tion or attrition behavior can be a func- 
tion of unmeasured and unmeasurable 
variables. Setting up an experiment 
may reduce the (largely unknown) ex- 
tent of the selection bias problem, but it 
will not eliminate it, since some form of 
attrition will always occur, human sub- 
jects being what they are. Thus, these 
authors are attacking, as many others 
have, a very difficult and significant 
problem — perhaps the problem in so- 
cial program evaluation. 

In their first chapter, “Methods Used 
to Evaluate Employment and Training 
Programs in the Past,” the authors set 
forth the history of nonexperimental 
methods used to attempt a correction 
for selection bias. This chapter will be 
particularly useful to those who wish to 
acquire a thorough review of the exist- 
ing state-of-the-art regarding these 
econometric procedures. 

Across the set of methods now used, 
such as the Heckman two-stage ap- 
proach, the crucial question is whether 
the model used to control for selection 
is statistically identified by any of the 
observed variables that are included in 
the model. Put simply, the two-stage 
Heckman approach first estimates a 
probability function to predict program 
participation. It then feeds the informa- 
tion gained from this probability func- 
tion into the benefit function used to 
measure program impact. Heckman has 
demonstrated that in a properly speci- 
fied model, if there is only one variable 
that identifies the probability function 
in the model, his procedure controls for 
the selection. Estimated net program ef- 
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fects are unbiased. Note that just add- 
ing more measured variables to the 
probability function in the first stage of 
the Heckman procedures is no guaran- 
tee whatsoever that identification has 
occurred. In short, the solution to the 
problem is elusive since variables such 
as motivation, IQ, or the taste for work 
are either unmeasurable or difficult to 
measure. One never knows for sure, giv- 
en the current state of statistical art, if a 
given model is identified in a behavioral 
sense. 

In effect, the authors argue in Chap- 
ter 2, “The Case for Applicant-Based 
Comparison Groups,” that the statistical 
identification problem is reduced if one 
selects as a comparison group those in- 
dividuals who in some way are "non- 
participating applicants”. The idea here 
is that such non-participating applicants 
are more similar to actual participants 
than are any other members of the pro- 
gram’s target group, in terms of unmea- 
sured characteristics. (Recall that the 
legislated target group for a given pro- 
gram is often very broadly defined, such 
as “the long-term unemployed” or “the 
economically disadvantaged”.) The au- 
thors argue that this approach is partic- 
ularly valuable if systematic records of 
the observable characteristics of appli- 
cants are kept throughout each step of 
the selection and screening process by 
the program managers who are selecting 
program participants. 

They properly note, however, that 
questions concerning unmeasurable 
characteristics such as motivation, in- 
herent ability, or the taste for work will 
still exist. So, a priori, while it is rea- 
sonable to assume that the authors’ rec- 
ommended procedure will reduce the 
problem of dealing with unmeasurable 
characteristics that underlie selection 
into a program, one does not know to 
what extent the problem is resolved. 

And in any case, the authors’ main as- 
sertion that non-participating applicants 
are those most similar to the program’s 
intended participants is always subject 
to empirical verification. 

Chapter 3 of the book, titled “Esti- 
mating Program Effects in the Home- 
maker-Home Health Aide Demonstra- 
tion,” presents the initial test of the 
authors’ recommendation in Chapter 2 
by using a database that has both a true 
control group and observations about 
applicants who were not allowed to par- 
ticipate or who voluntarily declined par- 
ticipation at some stage in the program 
operation process. The evaluation of 



this demonstration is a true experiment 
that includes the use of a subjective 
rating index by the program’s intake 
workers concerning an applicant’s suit- 
ability to be a homemaker-home health 
aide. The rating scale ranges from 1 (ex- 
cellent prospect) to 4 (poor prospect). 
Detailed data on measured personal 
characteristics are supplemented with 
two years of preprogram earnings data 
and four years of postprogram earnings 
data from Internal Revenue Service files. 
Consequently the database contains 
highly accurate earnings data — in the 
context of a true experiment with some 
attrition — and an attempt to subjec- 
tively assess certain unmeasurable char- 
acteristics that might predict program 
success. The study sample contains: 

■ Withdrawals: persons who applied 
but dropped out before they could be 
screened by program operators. This 
group contains individuals who may 
or may not have been selected into 
the program. 

■ Screen-outs: persons who applied 
but were rejected for admission to 
the program. 

■ No-shows: applicants accepted for 
participation but who failed to par- 
ticipate. 

■ Experimental participants. 

■ True controls. 

To test the efficacy of their recom- 
mendation concerning comparison 
group selection, the authors first esti- 
mated net program impacts with the ex- 
perimental participants and the true 
controls. Then they compared these re- 
sults against separate regression models 
that use as the comparison group the 
withdrawals, the screen-outs and the 
no-shows, respectively. One might ex- 
pect, a priori, that the withdrawals and 
the screen-outs would not measure up 
well as valid comparison groups. This 
turned out to be the case. Neither were 
the results encouraging for the no- 
shows, even after employing in the sta- 
tistical models the selection rating index 
scored by program operators. 

It is arguable that the no-show group 
is the most comparable to the treatment 
group. However, this statement depends 
entirely on why the no-shows do not 
participate, and on the variety of rea- 
sons the no-shows do not participate, 
since they are not likely to be a homo- 
geneous group.The reasons for selection 
out of the program are key. If these rea- 
sons cannot be measured, then the effi- 



cacy of using no-shows as a comparison 
group is in considerable doubt. This 
group may serve as a comparison group 
no better than any other group of pro- 
gram-eligible individuals. 

To see how matters turned out in this 
case, the empirical results are worth dis- 
cussing. The initial difference in mean 
earnings over the four-year follow-up 
period represented by the IRS earnings 
income data — that is, experimental 
earnings minus no-show group earnings 
— ranged from $1,039 (year two) to 
$1,192 (year three). Regression adjust- 
ment that included the subjective rating 
index reduced the postprogram gain 
from a low of $135 (year one) to a high 
of $333 (year two) But these results 
were still $123 to $420 higher than the 
true experimental net effects, which 
ranged from $643 (year one) to $394 
(year three). Since the value of the so- 
cial investment criterion (the internal 
rate of return) depends on the absolute 
size of the cost and benefit effects over 
time, there is the potential for consider- 
able bias in the estimation of this in- 
vestment measure if one uses the no- 
shows as a comparison group. 

The authors attempted to overcome 
the difficulty of having to make an arbi- 
trary choice among these estimated 
models and their comparison groups in 
Chapter 5, “Testing Alternative Methods 
for Selection Bias”. The main question 
addressed is “Which, if any, of the ap- 
plicant-based approaches is sufficiently 
protected from selection bias to provide 
adequate substitutes for a randomized 
experiment?” The authors provide an ef- 
fective review of the literature that fo- 
cuses on the attempt to select an unbi- 
ased estimator. They then move on to 
their preferred method, the develop- 
ment of a Bayesian decision framework 
that will allow them to distinguish 
among competing estimates of net pro- 
gram impact. 

While clear, the discussion is in- 
volved. It will most likely escape the in- 
tellectual appreciation of most govern- 
ment policymakers, legislators and their 
staffs, based on this reviewer’s own ex- 
perience over the past thirty years. Be- 
ing difficult to explain and quite closely 
and logically reasoned, the discussion 
will probably fail to meet the authors’ 
own subjective test of an effective policy 
tool. In their own words, “It is not 
enough that an estimation technique be 
considered free of selection bias in the 
minds of the researchers; it must also 
be viewed (the emphasis is in the origi- 
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nal) as bias-free by policymakers if its 
results are to be trusted and used.” 

Even more to the point, as the au- 
thors point out, "...even if one or more 
of the applicant-based estimators exam- 
ined here passes the validation tests we 
employ, we cannot claim to have found 
an applicant-based technique that will 
be as reliable as an experiment in all 
contexts.” The method proposed has 
high internal validity but unknown ex- 
ternal validity. What has eternally elud- 
ed us is the discovery of an estimator 
with a high level of external validity 
(which allows us to generalize net im- 
pact results), or a statistical test that 
will identify such an estimator from 
among a competing set. 

Thus readers are left with a sense of 
frustration in assessing the authors’ ef- 
forts. Nevertheless, despite this continu- 
ing problem, the monograph is a very 
thoughtful and well executed analysis. 

It should be required reading for every- 
one performing program analyses, for 
those attempting to interpret the results 
of such analyses, and for those who de- 
sign new social initiatives. 



O 




■■ 

Participatory Mixed- 
Method Evaluation 



Many youth experts have claimed 
that narrow sets of issues and singular 
evaluation methodologies, particularly 
in isolation from one another, are totally 
inappropriate for studying youth devel- 
opment, education, training and em- 
ployment issues. They insist that a 
broad set of issues must be addressed 
simultaneously, a continuum of re- 
search designs applied, and multiple 
methods used if we are to gain an in- 
depth understanding of the relationship 
between 1) social program interventions 
and 2) program outcomes and longer- 
term effects in social policy initiatives 
targeted to adolescents and young 
adults. This point of view stems from 
the belief that the interaction between 
socialization experiences and institu- 
tional interventions are exceedingly 
complex — young people are in a period 
of self-image and behavioral evolution, 
vulnerable to an array of influences and 
choices. Consequently these experts 
propose that the evaluation of youth 
programs must address a wider array of 
developmental and other issues, 
through the use of a more "participa- 
tory,” "mixed method” type of research. 

Even Nobel laureates in Economics 
have given attention to less mainstream 
approaches. For example, Herbert Si- 
mon, in The State of Economic Science: 
Views of Six Nobel Laureates, published 
in 1989 by the W.E. Upjohn Institute for 
Employment Research said: 

A very sophisticated body of 
econometric theory and method- 
ology has been created since 
World War II to deal with the 
problem of carrying out economic 
analyses with data that are too ag- 
gregated, measure the wrong con- 
cepts, are too infrequently collect- 
ed, and are exceedingly noisy. The 
experience of the past 50 years 
has shown that these sophisticat- 
ed econometric tools do not com- 
pensate for the poor quality of the 
data to which they are applied. 

...Other kinds of data can of 
course be obtained: data that tell 
us something about the expecta- 
tions, preferences, and choices of 
economic actors at micro levels. 



Opinion and attribute studies can 
be conducted. ..Actual decision- 
making processes can be 
observed.. .but polling data and 
"case studies” of individual be- 
havior are mistrusted in orthodox 
economic methodology. The use 
of these kinds of data to discover 
the realities of economic life and 
to provide an empirical basis for 
the auxiliary assumptions we 
need will require a revolution in 
methodological beliefs in econom- 
ics, and a revolution in the train- 
ing of economists... 

An International Exemplar 

The Evaluation Development and Re- 
view Unit of The Tavistock Institute in 
London, directed by Elliot Stern, who 
organized the United Kingdom’s evalua- 
tion association, has been applying par- 
ticipatory mixed-method evaluation ap- 
proaches for over twelve years. What 
the Tavistock researchers have done is 
to weld together, with strong bonding at 
the front end of the evaluation process, 
the following ingredients: 1) the princi- 
ples and practices imbedded in plan- 
ning, systems analysis, organizational 
theory and human resource develop- 
ment, and 2) their research training, 
knowledge and expertise. What further 
distinguishes their approach from tradi- 
tional practice is the synthesizing of the 
norms, values and ethical contexts that 
support these other disciplines. This 
borrowing, marrying and integrating of 
different perspectives has led them to 
embrace a collaborative, supportive, 
mixed-method approach to evaluation. 

A new wave of American evaluation 
researchers also have emphasized the 
importance of the relevance of an evalu- 
ation to its policy context; the appropri- 
ateness of an evaluation plan to the 
goals and developmental phase of the 
program being evaluated, and to its en- 
vironment; and the competence and ob- 
jectivity of the evaluator’s methodology. 
An appreciation of the political, socio- 
cultural, economic and organizational 
environment of an evaluation effort is 
viewed as a critical asset. An ability to 
invest key players from the beginning to 
the end of the evaluation planning, im- 
plementation and utilization process is 
understood to be essential in encourag- 
ing and sustaining support for an evalu- 
ation activity. It is considered signifi- 
cant in assuring that evaluation results 
and recommendations are communicat- 
ed honestly, communicated at the right 
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decision points in the policymaking pro- 
cess, and communicated to those in a 
position to initiate, rationalize and 
maintain change.This is a stakeholder, 
user-oriented evaluation process that 
demands an eclectic tool kit. 

A side benefit of this approach, not 
insignificant, is what Americans now 
like to call "capacity-building.” Building 
greater evaluation capability recognizes 
and appreciates the differences in the 
assumptions, values, knowledge base, 
training and experience of the diverse 
constituencies involved in most evalua- 
tion efforts, and seeks to reduce these 
differences by encouraging the partici- 
pation of diverse actors while providing 
them with an education about the re- 
search process. 

In the wake of an unprecedented de- 
centralization of social programs across 
all postindustrial nations in the context 
of new economic restraints, increasing 
local and regional capability and trust 
are now mandatory. The pay-off is new 
possibilities at those levels for exploring 
subnational questions of direct interest 
and concern to people responsible for 
operating and modifying programs. 

Strengths and Weaknesses 

Stern commented about the potential 
weaknesses, as well as the clear 
strengths of the more pragmatic mixed- 
method approach, in his session at an 
international evaluation research confer- 
ence in Vancouver, British Columbia in 
November 1995. The following com- 
ments by the editor of Evaluation Fo- 
rum, Ann Bonar Blalock, echo some of 
Stern’s anxieties and suggest others: 

1. This approach is more appropriate 
and useful in evaluating innovations 
in their early stages, because one 
wants maximum flexibility in this 
case for identifying key variables that 
should be measured and studied in 
later evaluations. But this approach 
should not be precluded in studying 
ongoing programs that have a more 
established track record, because it 
expresses a commitment to looking 
at both program implementation and 
effects. Large-scale, periodic evalua- 
tion efforts with ongoing programs 
could clearly benefit from a similar 
emphasis. 

2. It appears to be true that internal 
self-evaluation efforts (part of the 
participatory approach) that are tied 
carefully to less subjective indepen- 
dent evaluation activities can best in- 



vest program practitioners and their 
stakeholders in the evaluation pro- 
cess, increase the often troubled 
working relationship between re- 
searchers, program staff and others 
invested in the program being evalu- 
ated, and increase substantially the 
use made of evaluation information 
for continuous program improvement 
and policy change. 

The concern, however, is that it is a 
sizable challenge to maintain suc- 
cessful coordination between 1) self- 
evaluation activities and results, and 
2) evaluation activities conducted by 
external and program-independent 
researchers and their results. This re- 
quires an external team with the 
funding, time and diverse expertise 
to assure that joint planning at the 
front, collaborative implementation 
in the middle, and an integration of 
results and recommendations at the 
end are held together : Where the 
needed resources for useful self-eval- 
uations appear to be unavailable, 
where the management of this com- 
plex an evaluation effort is likely to 
be weak, or where the independent 
evaluators commissioned for the 
study are likely to take inappropriate 
control of the entire effort, there are 
problems. 

3. The importance of developing a new 
"evaluation culture” cannot be over- 
emphasized. This is what the Tavis- 
tock approach is all about. It requires 
a redefinition of the traditional roles 
of policymaker, program operator, 
stakeholder and researcher in the 
collaborative, information sharing di- 
rection — a long overdue revision of 
evaluation researchers’ usual style 
and inclination. 

4. The Tavistock researchers point out 
that the relationship between 1) pro- 
gram safety in a hostile world, and 
2) the willingness of program per- 
sonnel to resist the demand to "look 
good at all costs” is a significant one 
in this era of economic scarcity and 
government downsizing. But one 
wonders if the collaborative mixed- 
method approach can survive this 
onslaught, since program staffs tend 
to feel now like the workers caught 
in a web of potential mergers and 
mass layoffs in the corporate sector. 
If evaluators can be seen as safety 
mechanisms rather than as danger 
zones, there will be continuing sup- 
port for a better approach — but this 
remains to be seen. And the drying 
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up of research funding may eclipse 
this problem altogether. 

5. The Tavistock approach is demand- 
ing of multiple skills and talents, and 
quality control and information inte- 
gration are large challenges not all 
evaluation teams could manage. The 
potential cost of this interdisciplinary 
expertise is also a consideration, par- 
ticularly the excellent introduction of 
educational training seminars, some- 
thing all researchers should try to in- 
corporate in evaluation efforts. 

6. The Tavistock approach involves tai- 
loring a unique mix of organizational 
and research strategies to the priori- 
ties given attention by policymakers, 
and to the nature and development 
stage of a social initiative — some- 
thing researchers frequently fail to 
do in requesting and launching eval- 
uation projects. 

7. There are valid concerns about mak- 
ing comparisons across similar pro- 
grams at the local, regional, national 
and transnational levels, as part of 
the mix of methods. This is a very 
tricky business. One must reduce 
bias in such comparisons by identify- 
ing the criteria to be used in judging 
the true comparability of the pro- 
grams to be compared. Important is 

a prior description of conditions such 
as the following: 

- the policy context 

- the program environment 

- the intended program goals, 
means, interventions 

- the intended population to receive 
the interventions 

- the desired outcomes for this pop- 
ulation 

- the actual characteristics of pro- 
gram implementation 

- the actual characteristics of the 
target group 

- the actual interventions received 

- the characteristics of program 
management and service delivery 
and the nature of the information 
collection, storage, and analysis 
process 

- the research methods used and by 
what kinds of researchers 

The problem is that one cannot 
take comparability for granted. The 
title of the project or program may 
be the same from site to site, but 
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there are always significant differ- 
ences in the way an abstract pro- 
gram is given life in a particular en- 
vironment. These differences render 
sites less than comparable. Conse- 
quently evaluators must try to identi- 
fy and compensate for these differ- 
ences in comparing the 
implementation and effects of these 
different applications of a central 
concept, or of programs that appear 
to flow from the same premises and 
serve the same general purposes. 

8. Not apparently included in the mix 
of methodologies within the Tavis- 
tock approach are research designs 
which estimate the net effect of a 
program’s unique set of interventions 
— these experimental and quasi- 
experimental evaluation designs are 
generally considered "state of the 
art” impact methodologies by a large 
cohort of American evaluation re- 
searchers. The ability to estimate the 
net return on the public’s investment 
in a program has appealed to policy- 
makers in this market economy for a 
good reason — it allows them to 
place evaluation results within a fa- 
miliar investment/profit framework. 

But these designs are not perfect, 
therefore their absence in the mixed 
method approach may not represent 
a research deficit. Such methodolo- 
gies limit the number of variables 
that can be studied, and the number 
of measures that can be used to de- 
fine those variables. They alter the 
intended implementation mode of a 
program that may distinguish it from 
other alternatives and help explain 
its effects. It is difficult to convince 
program staff that maintaining ran- 
dom assignment procedures is criti- 



cal to the evaluation. The treatments 
are often poorly defined. And it is 
difficult to interpret the results of ex- 
periments because net impact stud- 
ies are frequently authorized in the 
absence of an effort to study the 
characteristics of program implemen- 
tation that may explain program ef- 
fects. Even highly accurate informa- 
tion about outcomes can be 
misleading in the policy process, if 
one has no idea why or how they 
happened. 

9. In this complex an effort, researchers 
must keep track of who , with what 
potential set of biases, is collecting 
information about what and by what 
means, so that the evaluation team 
holding this all together can distin- 
guish between different levels of sub- 
jectivity in the information to be 
brought together at the end. This is 
critical in interpreting information 
from multiple sources and therefore 
in making policy and program rec- 
ommendations based on this infor- 
mation. 



Editor’s Note 



Elliot Stern’s paper at the interna- 
tional conference, written with Elizabeth 
Sommerlad, was titled “The Devolved 
Evaluation of Dispersed Programmes.” 
Stern is the editor of a new internation- 
al evaluation research journal pub- 
lished by Sage titled Evaluation. 

An interesting article in the fall 1995 
newsletter of Harvard University’s Fami- 
ly Research Project, The Evaluation 
Exchange , discusses an approach sim- 
ilar to Tavistock’s in “Participatory Eval- 
uation: Enhancing Evaluation Use and 
Organizational Learning Capacity.” The 



authors propose that participatory eval- 
uation is an extension of the traditional 
stakeholder-based model. Key charac- 
teristics are: 

■ Control of the evaluation project is 
jointly shared by researchers and 
practitioners. 

■ There is a limit to the number of 
stakeholders involved. 

■ There is an unusual depth of partici- 
pation among non-researchers. 

In the traditional model, stakehold- 
ers are consulted early to define the fo- 
cus of an evaluation and later to help 
interpret the data, but they are only 
rarely involved during the research pro- 
cess and write-up phase. In the partici- 
patory model, joint control of the evalu- 
ation means that the evaluation agenda 
is mutually determined and conducted. 
Stakeholder participation, however, is 
limited to those with program responsi- 
bility or an investment in the program. 
Members of the program community 
are involved in defining the evaluation, 
developing instruments, collecting data, 
processing and analyzing data, and re- 
porting and disseminating results. This 
involvement of practitioners and stake- 
holders is expected to increase the util- 
ity of an evaluation and the capacity of 
those participating to organize and car- 
ry out useful evaluations. 

The senior author of the article, J.B. 
Cousins of the University of Ottawa, 
Canada, has written extensively on this 
approach to evaluation. He also has 
written two papers for international re- 
search conferences which may be of in- 
terest: “Collaborative Evaluation: A Sur- 
vey of Practice in North America,” and 
“Consequences of Researcher Involve- 
ment in Participatory Evaluation. “ 
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The European 
Commission’s Effort 
to More Effectively 
Structure Evaluation 
Activities 

The directorates of the European 
Commission (EC), the central govern- 
ment of the European Union, carry out 
the policies developed by the executive 
bodies of the Union, the Council of Min- 
isters and the European Parliament. A 
significant proportion of the Commis- 
sion’s funds are the Structural Funds 
for large-scale economic development, 
employment and training, and other hu- 
man resource projects and programs 
across member states of the Union. Al- 
though the Commission has required 
that the use of these funds be moni- 
tored and periodically evaluated, the in- 
formation made available to the EC 
from past assessments has not been 
viewed as fully adequate for decision- 
making. 

In an environment of increasing con- 
cern about establishing the European 
Commission’s accountability in a de- 
volved and decentralized social provi- 
sion system, the Commission recently 
sponsored a special project to enhance 
the evaluation of projects under the 
Structural Funds. This effort emphasizes 
the concept of Intermediate Evaluation 
— the ongoing and systematic evalua- 
tion of these projects and programs, as 
compared with previous periodic, often 
one-time-only evaluation efforts. The 
Center for European Evaluation Exper- 
tise in Lyon, France won the contract to 
organize this project. 

The Development of a New Set 
of Guidelines for Evaluation 

The Center sponsored several semi- 
nars in Brussels in 1995 on evaluation 
planning, design, implementation and 
utilization. The seminars brought to- 
gether a select group of experts on 
evaluation research and project admin- 
istration to advise the EC — that is, 
evaluation research experts from the 
U.S. and member countries of the EU, 
administrators of relevant EC director- 
ates, and researchers on contract to the 
EC. The purpose of the presentations 
and structured discussions was to pro- 



vide insights that could be incorporated 
by the Center within a general evalua- 
tion guide for use by the Monitoring 
Committees the EC had established to 
be formally responsible for planning 
and oversight regarding the projects 
supported by the Structural Funds. 

These Monitoring Committees are 
unusually large, composed of represen- 
tatives of the EC, government officials 
in the host member countries, and 
Structural Fund project managers and 
program operators in the areas served 
by the Committees. The staffs support- 
ing these committees have not, in most 
cases, included personnel with research 
training or experience. Consequently the 
Committees’ mandate to evaluate has 
been unevenly honored, with each 
Committee tending to impose its own 
definition of evaluation. 

The intent of the new guide was to 
increase the coordination of evaluation 
approaches, policies and activities with- 
in the Monitoring Committees around a 
standard set of principles and recom- 
mended practices. This standardization 
of a core understanding of the value 
and nature of Intermediate Evaluation 
was expected to assure more efficient 
and effective studies of projects, and a 
greater use of findings in improving the 
results of initiatives that flowed from EC 
policies. 

Following the seminars, the begin- 
ning guide, Organizing Intermediate 
Evaluation in the Context of Partner- 
ships , was published in late 1995. The 
word “partnership” has special mean- 
ing, since the European Union (EU) in- 
volves a complex set of vertical and hor- 
izontal partnerships pursuing common 
goals, which are reflected in the struc- 
ture of the Monitoring Committees. In 
this sense, the EU is a great experiment 
in testing the viability of building coop- 
erative relationships among powerful 
sovereign partners. The Monitoring 
Committees are a potentially exciting 
test of collaborative evaluation planning 
and utilization. 

In this context, the evaluation guide 
is expected to raise partners’ awareness 
of effective strategies for organizing and 
managing competent evaluations with 
high utility. The guide does not, howev- 
er, suggest that an incentive/sanction 
system be developed to secure compli- 
ance with the guide’s positions and rec- 
ommendations. 
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Partnership Collaboration and 
the Goals of Intermediate 
Evaluation 

The EC guide places strong emphasis 
on a heightened sensitivity to the needs, 
interests and resources of multiple 
stakeholders, and the creation of a col- 
laborative partnership-oriented environ- 
ment — a new evaluation culture. It 
differentiates between “mature” part- 
nerships characterized by shared objec- 
tives and combined budgets; “single ac- 
tor-dominated” partnerships that are 
more hierarchical; and “loose ad hoc” 
partnerships that survive on weak con- 
sensuses. It asks that Monitoring Com- 
mittees appreciate these differences in 
developing stronger partnership net- 
works. Expanding the network further, 
the guide suggests that Monitoring 
Committees include representatives of 
the recipients of Structural Funds 
projects. Given this broad focus on col- 
laboration, the guide describes a dual 
purpose for evaluation activities: 
project/ program improvement, and the 
establishment of accountability with 
funders, partners and program recipi- 
ents. 

Because of variability in the compo- 
sition of Monitoring Committees, the 
guide outlines both 1) basic Intermedi- 
ate Evaluation requirements, and 2) op- 
tional recommendations. 

The basic requirements suggested are 
these: 

■ A clearly defined five-year evaluation 
plan. 

• The development of relevant targets 
to be accomplished over the five 
years. 

■ The specification of the outputs that 
have already been achieved. 

■ The evaluation of the impact of the 
“ evaluable " elements of projects. 

■ The development of recommenda- 
tions for corrective action or enhance- 
ment. 

It is acknowledged in the guide that 
the level of rigor of impact evaluations 
will inevitably be influenced by the in- 
terests, timelines and resources of par- 
ticular Monitoring Committees. Howev- 
er, there is no specific discussion of 
different kinds of outcome evaluations, 
such as simpler pre-post studies, or 
more complex quasi-experimental and 
experimental net impact studies. Nei- 
ther are there cautions about undertak- 
ing comparative studies across projects 
or sites where it is difficult to obtain 
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comparable baseline and outcome data. 
However, the guide seems to encourage 
Monitoring Committees to move toward 
rigorous net impact studies. 

The guide distinguishes among three 
kinds of evaluations: ex ante, intermedi- 
ate and ex post studies. In the European 
vernacular, ex ante studies typically 
provide baseline information at the be- 
ginning of a project or program and in- 
formation about their early implementa- 
tion. Ex post studies look at short and/ 
or longer-term outcomes at the end of a 
project’s or program’s official life span. 
Intermediate evaluation represents on- 
going evaluation of a project or program 
over its administrative or legislative life, 
and is expected to contribute informa- 
tion on the basis of which competent ex 
post evaluations can be conducted. Oth- 
er distinguishing features of ongoing 
evaluation in the guide are the length of 
time allotted for the evaluation process, 
the close association of evaluation activ- 
ities with the monitoring process, and 
the methodological difficulties in study- 
ing longer-term project or program ef- 
fects prior to the end of the life span of 
the initiative. 

Steps in Carrying out a New 
Evaluation Mandate 

The guide is organized as an ideal- 
type progression of steps. Monitoring 
Committees are expected to mirror 
these stages at their own pace and ac- 
cording to their own assessment of 
what steps can occur in logical order 
versus simultaneously. The overarching 
principle is that evaluation activities 
should be an integral part of Commit- 
tees’ ongoing responsibilities rather 
than a sometimes add-on. The steps are 
these: 

■ Securing the evaluation mandate and 
organizational support for evalua- 
tion. 

■ Setting priorities among evaluation 
issues and developing RFPs. 

■ Developing specific evaluation plans. 

■ Managing evaluation activities and 
maintaining their quality. 

■ Disseminating evaluation results 
with the goal of utilization by key ac- 
tors. 

These steps highlight features of on- 
going evaluation efforts that are intend- 
ed to produce unbiased information 



about the major aspects of projects and 
programs in a collaborative, inclusive 
atmosphere that supports utilization. 

Developing a Framework for 
Evaluation 

Monitoring Committees are encour- 
aged to develop a general, pragmatic, 
written “evaluation mandate” that clari- 
fies a particular evaluation’s orientation 
and priorities and the responsibilities of 
key actors. Apart from organizing evalu- 
ations more effectively, preparation of a 
mandate is expected to encourage dis- 
cussions across the vertical and hori- 
zontal partnerships within Committees. 
Such discussions are intended to moti- 
vate members to commit themselves to 
the objective of an evaluation, to a gen- 
eral set of research questions, to who is 
responsible for organizing and carrying 
out an evaluation, and to ways in which 
the results can be disseminated and 
used. Evaluation design seminars or re- 
treats are recommended as ways to 
structure this planning effort. 

Key issues to be decided are opera- 
tional responsibility, the adequacy of 
the evaluation budget, the timing of 
evaluation activities, the composition 
and roles of the evaluation team to be 
selected, and where and when the re- 
searchers are to be brought into the 
evaluation process. 

Having reached agreement on a man- 
date, Monitoring Committees are en- 
couraged to work toward a consensus 
on a core set of clearly conceptualized, 
realistic, "evaluable” questions, while 
appreciating the diverse interests and 
agendas of members. These questions 
can involve studies of intersecting poli- 
cies and programs, or studies that seek 
to isolate the influence of a particular 
set of interventions. The qualification is 
that the choice should be made explicit 
at the front-end of evaluation planning. 

Developing an Organizational 
Environment for Evaluation 

Some Monitoring Committees cur- 
rently leave evaluation planning to their 
staffs or to a group of program manag- 
ers. The guide strongly recommends 
that a new partnership needs to be 
formed, involving members of the Mon- 
itoring Committee, their staff, and an 
independent evaluation team. Creating 
an evaluation subcommittee, evaluation 
task force, or research advisory group is 
recommended, to assist Monitoring 



Committees in honoring their oversight 
responsibilities. Entities such as these, 
linked directly to evaluation, can pro- 
vide advice at the beginning of evalua- 
tion planning, review the progress of 
evaluation activities, suggest solutions 
to research and organizational problems 
that arise, and contribute insights about 
ways to present findings for use by dif- 
ferent audiences. Defining lines of au- 
thority and communication, and the 
roles to be played by such bodies, are 
considered a critical task. 

Identifying Program Assumptions 
and Objectives in Developing the 
Research Questions 

One of the most difficult early evalu- 
ation tasks is addressed in the guide — 
identifying the hierarchy of objectives 
and intended cause-effect relationships 
that define a particular project or pro- 
gram. The explicit and implicit hypothe- 
ses that are revealed in the legislation or 
directives initiating social programs 
tend naturally to be modified as these 
programs are operationalized. Conse- 
quently the development of evaluation 
questions must give attention to both 
the original program design and the op- 
erational program. The guide suggests 
that the results of such negotiations 
over goals, objectives and desired out- 
comes should be incorporated within 
Requests for Proposals (RFPs). 

The Selection of Evaluation 
Approaches and Methodologies 

The guide recommends that the bod- 
ies under, or associated with Monitoring 
Committees that are dedicated specifi- 
cally to evaluation, assume responsibili- 
ty for developing RFPs for evaluations 
and selecting evaluation teams with the 
approval of the Committees. The RFP 
development process is to resist promis- 
ing too much information about too 
many issues, given evaluation budgets, 
timelines and expertise. In this develop- 
ment process, Committees are encour- 
aged to consider 1) what kinds of evalu- 
ators are needed for a particular 
evaluation, given the research questions 
negotiated and incorporated in the 
RFPs, and 2) the need to contract with 
researchers whose training and experi- 
ence combines research expertise with a 
substantive knowledge of the type of 
project to be evaluated. 
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The Use of Indicators 

The EC has emphasized the use of 
“indicators” in making assessments of 
projects, and the guide includes impor- 
tant suggestions about the use of such 
indicators. It suggests, for example, that 
indicators must be key measures of ma- 
jor relationships expressed in a pro- 
gram’s design — therefore it is impor- 
tant to develop measures of variables 
describing program implementation as 
well as those representing priority out- 
comes. It is recognized that measures of 
some of these key variables may not be 
available to evaluators through adminis- 
trative data systems, and therefore must 
be defined and collected by the evalua- 
tion team. Some of the cautions in the 
guide regarding the development and 
use of indicators are these: 

■ A careful review of program objec- 
tives can assist Monitoring Commit- 
tees in deciding what indicators are 
critical to use in a particular evalua- 
tion. 

■ An effort should be made to develop 
measures of both shorter-term and 
longer-term outcomes, and both nar- 
rower and broader effects. 

■ Using both qualitative and quantita- 
tive indicators is important, particu- 
larly in studying the quality of out- 
comes and effects. 

Absent from the discussion of indicators 
is any reference to the use of perfor- 
mance standards, other than the inter- 
est expressed in general (performance) 
targets. 

The Independence of Evaluators 

The guide gives appropriate attention 
to the issue of research independence, 
cautioning Monitoring Committees to 
select evaluators who are not linked to 
the management of the program to be 
studied, have not been involved in its 
design or implementation, and were not 
part of applying recommendations from 
previous evaluations. This is an impor- 
tant caveat since a substantial portion 
of EC evaluations have been internal 
evaluations conducted by EC or member 
country civil servants in EC directorate 
evaluation units or program divisions. 
While supporting the idea of external 
evaluators, the guide recognizes the sig- 
nificant benefits in developing and 
maintaining a positive working relation- 



ship between evaluators, program staff 
and a program’s information system 
personnel. This is expected to increase 
the accessibility and use of monitoring 
information in evaluations, depending 
on data coverage and reliability. But the 
guide clearly reserves the design of 
analyses and the conducting of these 
analyses for the evaluation team. 

Developing Evaluation Plans 

The guide clarifies that Monitoring 
Committees should resist turning over 
the responsibility for developing the 
plan for evaluating a particular set of 
programs or projects exclusively to the 
evaluators selected. Rather, close com- 
munication and collaboration are rec- 
ommended between the Committee and 
evaluators as the evaluation team trans- 
lates the content of an RFP into a spe- 
cific evaluation plan. A clearly defined 
plan will consider the interests of the 
various partners as well as the goals of 
the program to be studied, which then 
engages stakeholders in utilizing evalua- 
tion results. 

Again, in developing evaluation 
plans, the guide recommends that Moni- 
toring Committees and their evaluation 
bodies consider both implementation 
and outcome issues. This is consistent 
with the guide’s emphasis on beginning 
with what hypotheses and assumptions 
are imbedded in a program’s design. 

The cause-effect relationships these hy- 
potheses propose are often neglected in 
singular evaluations of outcomes or im- 
pact. Monitoring Committees may ob- 
tain accurate information about such ef- 
fects, but with little knowledge of why 
or how they occurred. Consequently the 
results obtained may be misinterpreted, 
and critical information for improving 
programs left unidentified. 

Because each major EC Structural 
Funds project typically involves a set of 
programs within it. Committees are 
faced with decisions about whether to 
.mount ongoing evaluations of an entire 
project, to give special attention to cer- 
tain programs, or to compare programs 
within a project or across program sites. 
The guide suggests that Monitoring 
Committees use a number of criteria in 
resolving this dilemma, such as the 
project’s stage of development and the 
potential utility to the Committees and 
the EC of obtaining information on en- 
tire projects as opposed to specific pro- 



grams within them. Again, evaluators 
and Committees must work together on 
these issues. 

Managing and Reporting on the 
Evaluation Process 

The guide terms the management of 
evaluations “a process of dialogue and 
adjustment.” This collaborative mes- 
sage is clear in these suggestions con- 
cerning the implementation of an evalu- 
ation: 

■ Systems need to be developed that 
provide ongoing information about 
the progress of an evaluation to key 
actors. 

■ Assurances are needed that the eval- 
uation is being implemented by the 
senior evaluators selected, rather 
than personnel under them. 

■ Communication channels between 
the evaluators and the sponsors of 
evaluation must remain open and 
utilized throughout the evaluation 
process and the writing of progres- 
sive drafts of evaluation reports. 

Information objectivity is a priority 
in the guide. It identifies the desirable 
characteristics of evaluations and their 
reports as the following: 

■ Reliability: are the data used in the 
evaluation reliable and is the analysis 
of these data objective? 

■ Methodological appropriateness: 

why was this methodology selected 
and does it remove potential sources 
of bias to the extent possible? 

■ Report clarity and coverage: does 
the evaluation report explain the 
methodology as well as the results, 
and in a concise but full and accu- 
rate manner? 

■ Separation of results from action 
recommendations: are recommenda- 
tions clearly distinguished from the 
reporting of evaluation results? 

Dissemination and Utilization 

The guide is unique in its recommen- 
dations for using evaluation information 
to improve programs while establishing 
credibility with multiple constituencies. 
These are the suggestions: 

■ Different kinds of dissemination 
strategies should be developed, re- 
sponsive to the interests and con- 
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cerns of the different groups of part- 
ners, without compromising the ac- 
curate presentation of evaluation re- 
sults and the methods used to obtain 
them. For example, Monitoring Com- 
mittees may want to format the eval- 
uation information differently for EC 
directorates and member states, 
members of the Committees, pro- 
gram administrators and managers, 
stakeholders and the public - and 
may wish to phase in this series of 
reports. 

■ The distribution of evaluation reports 
should be accompanied in some cas- 
es by opportunities for providing 
feedback from potential users, such 
as small conferences, seminar or dis- 
cussion groups. 

■ Dissemination can also be accom- 
plished through video presentations, 
press conferences and other non- 
written means. 



Comments on the EC Guide 

The handbook produced by the Cen- 
ter for European Evaluation Expertise 
represents a major new direction in as- 
sessing the EC’s Structural Funds 
projects. Because these funds are con- 
tributed and spent in the context of a 
powerful supra-national environment — 
one characterized by the necessity to 
accommodate diverse national, regional 
and local interests and organizational 
agendas — this beginning effort to sys- 
tematize and make evaluation thinking, 
planning, implementation and use more 
effective is a dramatic development. 

The Center, directed by Eric Monnier, 
organizer of the European Evaluation 
Association, has produced a clear, non- 
technically written, artfully summarized 
set of guidelines covering key issues in 
social program evaluation. It is appro- 
priate particularly for the multiple part- 
nerships involved in sponsoring, orga- 
nizing, contributing to and using 
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evaluations in making policy decisions 
and improving programs funded jointly 
by member nations of the EU. 

However, the guide’s use is not limit- 
ed to Europe. It has practical application 
at the state level in the United States. Its 
insights are relevant to the evaluation of 
the new statewide human resources sys- 
tems which are being developed in de- 
volved and decentralized environments 
involving multiple partnerships. Many 
of the same evaluation issues and di- 
lemmas addressed in the guide are be- 
ing dealt with across American states. 

The publication’s major contribution 
in this respect is its emphasis on both 
scientific competence and partnership 
collaboration throughout the evaluation 
process. It projects a “best practices’* 
model of a participatory, collaborative, 
mixed-method approach to evaluation 
research that clearly has implications 
for the American experience. 
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U.S. Youth Initiatives 
and Program 
Evaluations: 

Selected Examples 

In total, a vast amount of attention 
has been directed toward the resolution 
of youth education, training and em- 
ployment problems in the form of poli- 
cy analyses and evaluation research. 

But public policy has frequently ig- 
nored the information produced. In the 
late eighties and during the nineties 
American analysts often turned to Euro- 
pean models, which have invigorated 
American strategies for addressing 
youth problems. 

However, in this article we focus on 
selected examples of American analysis 
and research that have attempted to uti- 
lize the information accumulated from 
multiple sources in evaluating youth 
programs. Readers interested in more 
information on youth initiatives in edu- 
cation, training and employment may 
access additional examples later in the 
Resources Section. 

Congress and the Federal 
Agencies 

The Departments of Education, La- 
bor, and Health and Human Services 
(HHS) have sponsored an array of 
youth initiatives and have supported 
sophisticated evaluations of some of 
these demonstrations. We highlight two 
initiatives: a pilot project sponsored by 
Health and Human Services, and the 
school-to-work legislation involving Ed- 
ucation and Labor. 

Health and Human Services 

In 1986, the Family Support Admin- 
istration within HHS sponsored The 
Young Unwed Fathers Project. The 
project responded to the need to review 
existing information about the unwed 
father population, in the context of a 
clear relationship between the skill lev- 
el of young men and fatherhood. Ten 
policy analyses were commissioned by 



the project for review and discussion at 
a national symposium funded with the 
assistance of The Rockefeller Founda- 
tion. In 1987, a major report was au- 
thored by J. Smollar and T. Olms titled 
Young Unwed Fathers: Research Review , 
Policy Dilemmas and Options. The report 
drew these conclusions: 

■ Unwed fathers need to be held re- 
sponsible for their children and re- 
quired to fulfill the minimum obliga- 
tions of fatherhood — legally 
establishing paternity and paying fi- 
nancial support. 

■ Unwed fathers should be encouraged 
to develop a personal relationship 
with their children. 

■ These fathers’ needs and rights must 
be balanced against their responsibil- 
ities. 

■ These fathers need assistance and 
encouragement in fulfilling parental 
responsibilities in terms of acquiring 
occupational skills and employment. 

■ Acquiring skills and employment 
benefits fathers, children and society. 

■ Different sectors of society, and dif- 
ferent levels of government, must 
work together in encouraging re- 
sponsible parental behavior. 

School-to-Work 

The School-To-Work Opportunities Act 
of 1994 responded to years of concern 
about youth development, education 
and employment issues in the U.S. It 
benefitted from expert opinion and re- 
search on youth issues in the U.S., and 
numerous site visits to European Union 
countries whose youth programs have 
provided important lessons for Ameri- 
can youth programming. The Act pro- 
vides federal funding to develop state 
and local systems for assisting youth in 
making a successful transition from sec- 
ondary education to further education 
and skills training, and then into pro- 
ductive careers. Federal “seed money” 
has been received by eight states in 
1994 and nineteen in 1995. States re- 
ceiving seed funds are expected to 
award subgrants to local school-to-work 
partnerships. 

The Act requires the Departments of 
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Education and Labor to conduct a na- 
tional evaluation of the systems devel- 
oped under the legislation. In 1995, 
Mathematica Policy Research, Inc. 

(MPR) was selected as the principal re- 
search firm to design and conduct this 
evaluation. 

The evaluation is guided by four key 
objectives: 

■ The assessment of states’ progress in 
creating systems that change the way 
youth are educated and prepared for 
careers. 

■ The identification of promising prac- 
tices and barriers to developing new 
school-to-work systems. 

■ The determination of the nature and 
extent of participation in partner- 
ships and programs developed by 
employers, schools, students, public 
programs, educational institutions 
and/or others. 

■ The measurement of the outcomes of 
students in high school, postsecond- 
ary education and training, and em- 
ployment. 

The evaluation plan to be imple- 
mented between 1996 and 1999 re- 
sponds to these objectives through three 
major components: 

■ A mail survey of local school-to-work 
partnerships, to collect data on: 

- Composition of partnerships. 

- School-based and workplace- 
based activities involved. 

- Linkages to postsecondary op- 
tions. 

- Approaches for assessing stu- 
dents. 

- Levels of student participation. 

- Aggregate measures of transitions 
out of high school and into post- 
secondary options. 

■ Case studies: a sample of seven to 
nine states and forty-two local part- 
nerships, to collect data on: 

- Approaches in developing school- 
based, workplace-based and con- 
necting activities. 

- The availability and utilization of 
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the opportunities developed for 
students. 

■ A survey of students using question- 
naires and a telephone follow-up: 
thirty-two randomly selected partner- 
ships in the same states in which 
case studies will be conducted, to 
collect data on three cohorts of 12th 
grade students at three measurement 
points between 1996 and 2000 re- 
garding: 

- Changes in students' access to 
and participation in school-based 
and workplace-based activities 
under the new systems developed. 

- Changes in high school, postsec- 
ondary and employment out- 
comes achieved. 

The Act was clear about requiring 
both a process (implementation) study 
and an effectiveness study. In May, 

1996, MPR produced their initial report 
on the School-to-Work Opportunities 
Act systems, titled School-To-Work Im- 
plementation Progress: The State Per- 
spective in Early 1996, written by Alan 
Hershey, MPR's project director, and 
Linda Rosenberg. The report covered a 
number of process evaluation issues 
emerging from discussions with direc- 
tors of school-to-work systems in the 
twenty-seven states receiving imple- 
mentation grants. Among these issues 
were: 

■ The nature of state and local partner- 
ships. 

■ The nature and extent of partnership 
resources. 

■ How “development” and “implemen- 
tation” grants are being utilized. 

■ The funding process for local part- 
nerships. 

■ The priorities set in developing sys- 
tems. 

■ Implementation “best practices.” 

■ Problems and successes regarding 
the involvement of employers. 

■ The role of the national School-To- 
Work Office. 

The researchers found that the major 
areas in which states felt they had made 
substantial progress were creating col- 
laborative relationships, developing ca- 
reer guidance activities, building local 
partnerships, creating a common vision 
for the school-to-work transition, and 
marketing that vision. At the substate 
level, the key areas of improvement 
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were partnership development, involv- 
ing employers, and linking curricula 
across diverse areas. 

The most important barriers to devel- 
oping school-to-work systems were a 
lack of understanding, knowledge and 
expertise regarding the development of 
these systems; bureaucratic inertia, re- 
sistance and conflict; and difficulty in 
creating and sustaining cooperation and 
collaboration. Similar obstacles were 
encountered at substate levels. The 
twenty-seven states receiving the imple- 
mentation grants viewed the national 
office as helpful and supportive, but ad- 
vocated a more coordinated, consistent 
and persistent campaign for school-to- 
work systems by key national leaders, 
and urged more involvement by em- 
ployers. 

In February, 1996, the Departments 
of Education and Labor sponsored a 
roundtable titled Net Impact Evaluation 
of School-to-Work: Exploring Alterna- 
tives. Six papers were commissioned for 
the roundtable which addressed issues 
surrounding the development of a net 
impact design. Broad areas of agree- 
ment emerged from the day-long ses- 
sion with school-to-work experts and 
researchers: 

■ A net impact evaluation should not 
attempt to measure the impact of the 
entire system, but rather the impact 
of components of the system and the 
interaction among these components. 

■ In studying the school-to-work sys- 
tem in this way, there was support 
for using a typology of state/local 
systems based on a number of char- 
acteristics. 

■ The research design should include a 
study of implementation (already in 
progress), a quasi-experimental and 
observational study of outcomes, and 
a small number of randomized trails 
to test either planned variations in 
the system or basic assumptions un- 
derlying system development. 

■ The study of system outcomes 
should include the impact on em- 
ployers, through a special survey or 
some other research strategy. 

School-to-work systems were viewed 
as involving more than the conventional 
array of outcomes, to encompass 
whether youth begin to formulate con- 
crete career plans, and whether employ- 
ers restructure their career ladders and 
offer higher wage jobs to high school 
graduates. In June 1997, the Depart- 



ment of Labor published the results of 
the roundtable in Evaluating the Net Im- 
pact of School-to-Work: Proceedings of a 
Roundtable. 

Public/Private Ventures, lnc.(P/PV) 

Several years prior to the project 
sponsored by HHS, Public/Private Ven- 
tures, Inc. designed the Young Unwed 
Fathers Pilot Project, funded by the 
Charles Stewart Mott Foundation and 
other contributors. It was operated in 
six sites across the country. Program- 
ming in the sites expressed five key ele- 
ments: education; occupational training 
through JTPA; “fatherhood develop- 
ment” activities; and counseling and 
ongoing support, including services 
eighteen months beyond job placement. 

The evaluation of the pilot project in- 
dicated that the Fatherhood Develop- 
ment curriculum developed by P/PV 
was a successful element of the pilot, 
but it also revealed these challenges: 

■ JTPA eligibility requirements, based 
on households and income, some- 
times acted as a barrier to the enroll- 
ment of young fathers. 

■ The lack of stipends frequently 
forced participants to find jobs pre- 
maturely, simply to meet short-term 
economic needs. 

■ The project’s requirement that partic- 
ipants establish legal paternity tend- 
ed to retard recruitment. 

P/PV's policy analysis, programming 
and evaluation activities go well beyond 
fatherhood issues, however. They em- 
brace a range of youth development, ed- 
ucation and employment problems. 

Its organizational priority has been to 
address “two fundamental weaknesses 
in youth policy: 1) a lack of program- 
matic attention to youth’s basic needs 
for sustained adult attention, for safety, 
for recreation and learning activities, 
and for work that holds future promise 
for labor market success, and 2) a lack 
of evaluation approaches that capture a 
youth’s developmental progress.” 

The firm has developed a number of 
curriculum modules as part of its pro- 
gramming goals, such as the 25-session 
Fatherhood Development module, the 
60-session WORKmatters module, Work 
and Society, Crosswalks, and Life Skills 
and Opportunities. P/PV has also con- 
ducted an experimental evaluation of 
Big Brothers/Big Sisters programs that 
suggested that adult/youth mentoring 
relationships can reduce dramatically 
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initial drug use by teenagers in single- 
parent families, and improve school per- 
formance and peer and family relation- 
ships. Beginning evaluations of the 
Bridges to Work and WorkPLus demon- 
strations suggested it was possible to in- 
volve employers in enriching the work 
experience and career potential of poor 
youth. And the firm’s work with neigh- 
borhood groups in initiatives such as 
Plain Talk and Community Change for 
Youth Development clarified that ade- 
quate resources and strong adult com- 
mitment had a significant influence in 
poverty areas. 

In 1992, P/PV conducted a study of 
a set of projects sponsored by the firm 
and others which were focused specifi- 
cally on the effectiveness of mentoring 
as a youth program intervention — 
P/PV’s Adult/Youth Relationships Pilot 
Project , as well as Temple University’s 
Linking Lifetimes programs, Campus 
Compact’s college-based mentoring 
projects and a program sponsored by 
the I Have A Dream Foundation. A se- 
ries of reports were produced in 1992, 
including College Students As Mentors 
for At Risk Youth by J. Tierney and A. 
Branch, and Adult/Youth Relationships 
Pilot Project by J. Greim. These reports 
revealed the complexity of successful 
mentoring programs as well as their 
clear benefits. 

The Manpower Demonstration 
Research Corporation (MDRC) 

In the early 1990s, MDRC studied a 
number of innovative school-to-work 
transition programs around the country, 
and the role played by employers in 
these programs. In 1993, MDRC pro- 
duced a report following the study, The 
School-to-Work Transition and Youth Ap- 
prenticeship by T. Bailey and D. Merritt. 
The report focused on the youth ap- 
prenticeship model, the 4-H program, 
cooperative education, career academies 
and Tech Prep. 

Interest in youth apprenticeship 
models grew dramatically in the U.S. in 
the 1990s based on an admiration for 
the German apprenticeship system. The 
U.S. Department of Labor created an Of- 
fice of Work-Based Learning which 
launched a number of pilot projects to 
test apprenticeship-type programs, and 
foundations such as Jobs for the Future 
supported experimentation with appren- 
ticeship approaches. However, the Ger- 
man socio-economic environment dif- 
fered significantly from American 



society in that new apprenticeship strat- 
egies were needed. While including 
components common to European mod- 
els, such as workplace-based learning 
and credentialing, alternatives such as 
the academies and Tech Prep seemed 
more appropriate. 

MDRC found that the workplace- 
based training component was the most 
significant problem area. Involving 
American employers in a systematic 
way, to support apprenticeship models 
over time, has proven difficult. School- 
based enterprises and cognitive appren- 
ticeship demonstrated potential as op- 
tions, but the linkage between these 
options and the workplace have re- 
mained tentative. Consequently MDRC 
concluded that there were important 
barriers to developing and implement- 
ing a large-scale youth apprenticeship 
system in which a significant part of the 
learning was workplace-based. The firm 
suggested building on models such as 
academies and Tech Prep, incorporating 
incentives to encourage employer par- 
ticipation. 

In 1996, MDRC produced a report on 
its evaluation of the implementation of 
career academies, Career Academies: 
Early Implementation Lessons from a 
Ten-Site Evaluation, by J. Kemple and J. 
Rock. The evaluation focused on the 
characteristics of academies, student re- 
cruitment and selection, enrollment and 
attrition patterns, and the characteris- 
tics of students and teachers. Also in 
1996, MDRC disseminated its evaluation 
of Ohio’s Learning, Earning and Parent- 
ing Program (LEAP). LEAP responded to 
two major concerns: the long period 
teenage mothers remain on public assis- 
tance, and the high dropout rate for 
poor teenagers in the educational sys- 
tem. The evaluation covered twelve 
counties and was reported in a series of 
reports. The final report, Three-Year Im- 
pacts of Ohio's Welfare Initiative to Im- 
prove School Attendance Among Teenage 
Parents by D. Long et al, analyzes 
LEAP’S effect on school completion, em- 
ployment and welfare receipt for a sub- 
sample of teens three years following el- 
igibility for the program. 

The first evaluation report showed 
that LEAP was feasible to operate. The 
second found that the program prevent- 
ed some in-school teens from dropping 
out and brought some dropouts back 
into the system. The third showed that 
increased school attendance translated 
to a significant increase in school com- 
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pletion for teens enrolled in school 
when they were determined eligible for 
the program, but little gain for teens 
who had dropped out prior to eligibility 
determination. Overall, only a segment 
of the program population experienced 
positive outcomes. 

The Carnegie Corporation and 
the Southern Education 
Foundation 

In 1995, the Corporation’s Council 
on Adolescent Development produced a 
major report titled Great Transitions: 
Preparing Adolescents for a New Centu- 
ry. Its recommendations fell heavily on 
the side of stronger roles for the em- 
ployer community, essentially giving 
added support to school-to-work alter- 
natives. It highlighted the social costs of 
ignoring the educational and occupa- 
tional skill deficits of American young 
people. A similar theme was evident in 
the Southern Education Foundation’s 
1995 report on the work of its Panel on 
Educational Opportunity and Postsec- 
ondary Desegregation titled Redeeming 
the American Promise. The panel found 
significant remnants of segregation 
shaping higher education, reducing edu- 
cational opportunity for minority stu- 
dents. The report called for an educa- 
tional system from kindergarten through 
four-year colleges that was more stu- 
dent-centered and performance-driven, 
and for a more comprehensive, system- 
wide approach to equal educational op- 
portunity. 

U.S. Congress: Office of 
Technology Assessment (OTA) 

Another 1995 report is an important 
resource, Learning to Work: Making the 
Transition from School to Work. The re- 
port indicated that work-based learning 
can help students understand the rele- 
vance of academic studies to their later 
lives, assist them in exploring career op- 
tions, encourage them to acquire desir- 
able work attitudes and habits, contrib- 
ute to the development of occupational 
skills, and prepare them for lifelong 
learning. OTA explored a range of 
school-to-work alternatives and provid- 
ed these insights: 

■ Workplace learning is most likely to 
be effective when planned jointly by 
schools and employers, when fo- 
cused not only on occupational skill 
development but on personal devel- 
opment and a broad understanding 
of industry, when coordinated with 
the academic and occupational in- 
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struction of schools, and when stu- 
dents’ progress is evaluated frequent- 
ly. 

■ The effects of school-to-work sys- 
tems on student success and work- 
force productivity will require more 
than a decade to evaluate, though in- 
terim assessments of implementation 
and student performance can be use- 
ful in refining systems as they devel- 
op. 

■ Commitment and resources from par- 
ticipating businesses are imperative 
regarding planning and implement- 
ing a sequence of training, mentoring 
and supervised work experience that 
is coordinated with school-based in- 
struction. 

■ Employers have been reluctant to 
participate because of inadequate 
preparation of students for work- 
place assignments, the lack of assis- 
tance in preparing and implementing 
work-based learning, the costs of 
training and mentoring, the cost of 
wages, resistance on the part of man- 
agement and incumbent employees, 
and the absence of incentives for in- 
volvement. 



Editor’s Note 



For a provocative and useful analy- 
sis of youth issues, please see Gary 
Walker’s speech in P/PV’s newsletter 
for fall 1993, “Abandoning the Mechan- 
ic’s Model: A New Direction for Youth 
Employment Policy.” Walker’s “develop- 
mental model” involves 1) developing 
the theory and discipline necessary to 
make difficult choices about which ele- 
ments of youth programs should be im- 
plemented, 2) investing up front in de- 
veloping the content to be delivered to 
youth once choices are made, 3) repli- 
cating successful elements and pro- 
grams, 4) continually collecting evi- 
dence of effectiveness, and 5) working 
with the private sector to define appro- 
priate roles for employers. 
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History of the Journal 

Evaluation Forum was developed 
as part of a five-year national dem- 
onstration project, The JTPA Evalu- 
ation Design Project, funded be- 
tween 1 986 and 1 991 by the National 
Commission for Employment Policy, 
the U.S. Department of Labor, the 
IBM Corporation and The Ford Foun- 
dation. The purpose of the project 
was to increase the interest, knowl- 
edge and sophistication of employ- 
ment and training professionals at 
the state and local level regarding 
program evaluation: its scientific 
principles and methods, its practical 
applications in better understanding 
program implementation and impact, 
and its utility as a practical tool for 
adjusting policies and improving pro- 
grams. The project’s series of 
evaluation guides can be obtained 
through the ERIC system in public li- 
braries, or in book form from the W. 
E. Upjohn Institute for Employment 
Research. 

Evaluation Forum is a national 
journal growing out of the five-year 
project and developed specifically 
for state and local program practitio- 
ners in the employment and training 
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field. Its major objective is to com- 
municate information on policy is- 
sues and research activities at the 
national, state and local level which 
can inform judgments about the de- 
sign and effectiveness of various 
employment and training policies 
and strategies. Eight theme-oriented 
issues of the journal were published 
as part of the demonstration project. 

The New Evaluation Forum 

The new Evaluation Forum, of 
which this is the fourth issue, is be- 
ing sponsored and funded by the 
Office of Planning and Research, 
Employment and T raining Adminis- 
tration, U.S. Department of Labor. 

Its purpose is to disseminate useful 
information on evaluation issues, ac- 
tivities and results to those respon- 
sible for administering, planning, 
managing and overseeing employ- 
ment and training programs across 
the United States. Again, each issue 
of the journal is theme-oriented, and 
follows the same organizational for- 
mat as in the past. However, each 
issue has featured reviews by the 
editor and guest reviewer rather than 
articles written by practitioners and 
researchers. 



Reader Participation 

The content of the new version of 
the journal will cover the same kinds 
of materials as in the past: commen- 
taries, policy analyses; program re- 
views; reports on evaluation plan- 
ning, reports on analyses of 
monitoring data; research reports on 
program evaluations or policy re- 
search; books and articles on policy 
issues, programs and research; 
and other materials related to evalu- 
ation that would be of interest to 
practitioners. 
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